We have experienced very slow execution when running bin/magento setup:upgrade command on various production and staging environments.

For the majority of eCommerce websites, an upgrade process should be fast and smooth. In this article, I will show you how to overcome performance degradation when running upgrades for your Magento 2 store.

Magento 2 Upgrade Command

The bin/magento setup:upgrade command provides upgrades for Magento 2 application and executes database schema and data changes. The command checks all modules and its versions. If the version of a module is different from the version installed and stored in the setup_module database table, the command will perform upgrade operation. This will trigger execution of IntallSchema.php and UpgradeSchema.php classes of the module.

Elastic File System

Everything can work well without any performance issues when upgrading Magento 2 with latest changes. However, there might be a slow execution of setup:upgrade command in case production server is configured to read Media from Amazon Elastic File System.

Amazon Elastic File System (Amazon EFS) provides simple, scalable file storage for use with Amazon EC2. With Amazon EFS, storage capacity is elastic, growing and shrinking automatically as you add and remove files, so your applications have the storage they need, when they need it.

Elastic File System is recommended for Magento 2 production environment with two or more EC2 instances to distribute load across multiple servers.

There is nothing wrong with using Elastic File System except the fact that it is slower compare to Elastic Block System volume which can be attached directly to an EC2 instance. ELB can’t be used when you have 2 EC2 instances running.

Amazon EBS volumes are created in a particular Availability Zone and can be from 1 GB to 16 TB in size. Once a volume is created, it can be attached to any Amazon EC2 instance in the same Availability Zone. Once attached, it will appear as a mounted device similar to any hard drive or other block device. At that point, the instance can interact with the volume just as it would with a local drive, formatting it with a file system or installing applications on it directly. A volume can only be attached to one instance at a time, but many volumes can be attached to a single instance.

How UpgradeCommand Checks Directories

The Magento\Setup\Console\Command\UpgradeCommand class installs data provided in the UpgradeSchema.php classes from every module enabled. However before doing this, it checks file permissions with the help of Magento\Framework\Setup\FilePermissions::getInstallationWritableDirectories() method. List of writable directories is hard-coded inside the getInstallationWritableDirectories() method.

Class: Magento\Framework\Setup\FilePermissions

[php]
public function getInstallationWritableDirectories()
{
if (!$this->installationWritableDirectories) {
$data = [
DirectoryList::CONFIG,
DirectoryList::VAR_DIR,
DirectoryList::MEDIA,
DirectoryList::STATIC_VIEW,
];
foreach ($data as $code) {
$this->installationWritableDirectories[$code] = $this->directoryList->getPath($code);
}
}
return array_values($this->installationWritableDirectories);
}
[/php]

The Magento\Framework\Setup\FilePermissions::getMissingWritablePathsForInstallation() method checks all directories recursively and verifies that writable permissions are set before executing Data Install/Update operation.

[php]
public function getInstallationCurrentWritableDirectories()
{
if (!$this->installationCurrentWritableDirectories) {
foreach ($this->installationWritableDirectories as $code => $path) {
if ($this->isWritable($code)) {
if ($this->checkRecursiveDirectories($path)) {
$this->installationCurrentWritableDirectories[] = $path;
}
} else {
$this->nonWritablePathsInDirectories[$path] = [$path];
}
}
}
return $this->installationCurrentWritableDirectories;
}
[/php]

The whole point here is that pub/media directory is symlinked to a physical media directory stored at the Elastic File System storage. Even if by any chance you are planning to install additional media images on production environment the time required to complete installation is more important than the time required to check file permissions across all sub-directories in pub/media directory. In 99% symlinked media directory has proper write permissions this media directory is served by Elastic File System and mounted to an EC2 instance.

The bottom line of this post is the following. Checking EFS where you have Media is way to expensive and might take up to 1 hour when you have 8GB of media. Without EFS the upgrade process can run for about 3-5 minutes.

How to fix it

There are few ways on fixing long execution time of the setup:upgrade command. Since the majority of time it takes to check every single sub-directory in media which is stored under EFS my recommendation it to eliminate this check.

New Pronko\Performance\Setup\FilePermissions class extends existing Magento\Framework\Setup\FilePermissions class and overrides getInstallationWritableDirectories() method. In this method we are going to check whether path of the directory which should be writable is a symlink or real. In case it is a symlink, we suppose it is an EFS storage and we simply exclude the directory from further recursive is_writable() check.

Assuming that’s only Media directory is stored under EFS we are going to skip only 1 out of 4 directories. The static files however, can also be placed under EFS, but I won’t recommend doing this.

Here is our Pronko\Performance\Setup\FilePermissions class.

[php]
namespace Pronko\Performance\Setup;

use Magento\Framework\Setup\FilePermissions as MagentoFilePermissions;

class FilePermissions extends MagentoFilePermissions
{
/**
* Retrieve list of required writable directories for installation
* Exclude symlinks from isWritable recursive checks
*
* @return array
*/
public function getInstallationWritableDirectories()
{
parent::getInstallationWritableDirectories();
foreach ($this->installationWritableDirectories as $code => $path) {
if (is_link($path)) {
unset($this->installationWritableDirectories[$code]);
}
}
return $this->installationCurrentWritableDirectories;
}
}
[/php]

Next step is to add a preference for our the Magento\Framework\Setup\FilePermissions in di configuration. Since setup code is a different application with Symphony configuration, there is no way we can simply use our own di.xml configuration file to add preference as we always do. Having said, we have to modify <magento_root>/setup/config/di.config.php file and add our new preference:

[php]
return [
‘di’ => [
‘instance’ => [
‘preference’ => [
‘Magento\Framework\Setup\FilePermissions’ => ‘Pronko\Performance\Setup\FilePermissions’
],
],
],
];
[/php]

*There are other references listed in the original file, which I excluded here in the example.

Also, we have to modify additional configuration file which stores aliases for Setup Application. Remember, we are talking about Symphony application – Setup. Here are changes to the <magento_root>/setup/config/application.config.php file.

[php]
use Magento\Setup\Mvc\Bootstrap\InitParamListener;

return [
‘modules’ => [
‘Magento\Setup’,
],
‘module_listener_options’ => [
‘module_paths’ => [
__DIR__ . ‘/../src’,
],
‘config_glob_paths’ => [
__DIR__ . ‘/autoload/{,*.}{global,local}.php’,
],
],
‘listeners’ => [
‘Magento\Setup\Mvc\Bootstrap\InitParamListener’
],
‘service_manager’ => [
‘factories’ => [
InitParamListener::BOOTSTRAP_PARAM => ‘Magento\Setup\Mvc\Bootstrap\InitParamListener’
],
‘aliases’ => [
‘magentoframeworksetupfilepermissions’ => ‘Pronko\Performance\Setup\FilePermissions’,
]
],
];
[/php]

Next time you will run bin/magento setup:upgrade command your new version of the code as well as database will be updated almost instantly. Magento won’t be checking permissions of the pub/media directory which is in reality a symlink to a location stored in Elastic File System storage.

Let me know how you run your production environment in comments.