We are facing issues with the automation tasks. They are very slow and often freeze. According to our hosting partner this happens because these tasks use to much memory. This happens mostly when running the “Generate thumbnails” and “Rescan files” tasks. On version 8 we never had these issues. What is causing this? More important how can we fix this? Thank you.
Hi @EnoRm - can you paste some or all of your environment info? It would be helpful to know what kind of memory limits you’re working with.
Here are the system requirements just to compare vs. what you might have on the server:
1 Like
Concrete Version
Core Version - 9.2.1
Version Installed - 9.2.1
Database Version - 20230503095900
Environment
live
Database Information
Version: 10.6.15-MariaDB
SQL Mode: ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
Concrete Packages
Block Designer (4.1.4), Block Designer Pro (4.1.3), Enorm Blackbox (1.0.0), Enorm Connector (1.0.28), EnoRm Sitemap (1.0.1), Fileset Attribute (1.1), Formidable (1.1.4.4), Open Graph Tags Lite (2.1.6), Page List+ (4.2.1)
Concrete Overrides
blocks/buttons/add.php, blocks/buttons/auto.js, blocks/buttons/composer.php, blocks/buttons/config.json, blocks/buttons/controller.php, blocks/buttons/css_form/repeatable-ft.form.css, blocks/buttons/css_form, blocks/buttons/db.xml, blocks/buttons/edit.php, blocks/buttons/form.php, blocks/buttons/icon.png, blocks/buttons/js_form/handlebars-helpers.js, blocks/buttons/js_form/handlebars-v4.0.4.js, blocks/buttons/js_form, blocks/buttons/view.php, blocks/buttons, blocks/card_column/add.php, blocks/card_column/auto.js, blocks/card_column/composer.php, blocks/card_column/config.json, blocks/card_column/controller.php, blocks/card_column/css_form/repeatable-ft.form.css, blocks/card_column/css_form, blocks/card_column/db.xml, blocks/card_column/edit.php, blocks/card_column/form.php, blocks/card_column/icon.png, blocks/card_column/js_form/handlebars-helpers.js, blocks/card_column/js_form/handlebars-v4.0.4.js, blocks/card_column/js_form, blocks/card_column/view.php, blocks/card_column, blocks/contact_information/add.php, blocks/contact_information/auto.js, blocks/contact_information/composer.php, blocks/contact_information/config.json, blocks/contact_information/controller.php, blocks/contact_information/css_form/repeatable-ft.form.css, blocks/contact_information/css_form, blocks/contact_information/db.xml, blocks/contact_information/edit.php, blocks/contact_information/form.php, blocks/contact_information/icon.png, blocks/contact_information/js_form/handlebars-helpers.js, blocks/contact_information/js_form/handlebars-v4.0.4.js, blocks/contact_information/js_form, blocks/contact_information/view.php, blocks/contact_information, blocks/establishments_slider/add.php, blocks/establishments_slider/auto.js, blocks/establishments_slider/composer.php, blocks/establishments_slider/config.json, blocks/establishments_slider/controller.php, blocks/establishments_slider/css_form/repeatable-ft.form.css, blocks/establishments_slider/css_form, blocks/establishments_slider/db.xml, blocks/establishments_slider/edit.php, blocks/establishments_slider/form.php, blocks/establishments_slider/icon.png, blocks/establishments_slider/js_form/handlebars-helpers.js, blocks/establishments_slider/js_form/handlebars-v4.0.4.js, blocks/establishments_slider/js_form, blocks/establishments_slider/view.php, blocks/establishments_slider, blocks/hero/add.php, blocks/hero/auto.js, blocks/hero/composer.php, blocks/hero/config.json, blocks/hero/controller.php, blocks/hero/css_form/repeatable-ft.form.css, blocks/hero/css_form, blocks/hero/db.xml, blocks/hero/edit.php, blocks/hero/form.php, blocks/hero/icon.png, blocks/hero/js_form/handlebars-helpers.js, blocks/hero/js_form/handlebars-v4.0.4.js, blocks/hero/js_form, blocks/hero/view.php, blocks/hero, blocks/page_list_plus/controller.php, blocks/page_list_plus/templates/brand.php, blocks/page_list_plus/templates/slider.php, blocks/page_list_plus/templates/vacancy-card.php, blocks/page_list_plus/templates/vacancy-hero-filter.php, blocks/page_list_plus/templates/vacancy.php, blocks/page_list_plus/templates/vacancy_slider.php, blocks/page_list_plus/templates, blocks/page_list_plus/view.php, blocks/page_list_plus, blocks/testimonials/add.php, blocks/testimonials/auto.js, blocks/testimonials/composer.php, blocks/testimonials/config.json, blocks/testimonials/controller.php, blocks/testimonials/css_form/repeatable-ft.form.css, blocks/testimonials/css_form, blocks/testimonials/db.xml, blocks/testimonials/edit.php, blocks/testimonials/form.php, blocks/testimonials/icon.png, blocks/testimonials/js_form/handlebars-helpers.js, blocks/testimonials/js_form/handlebars-v4.0.4.js, blocks/testimonials/js_form, blocks/testimonials/view.php, blocks/testimonials, blocks/vacancy_categories/add.php, blocks/vacancy_categories/auto.js, blocks/vacancy_categories/composer.php, blocks/vacancy_categories/config.json, blocks/vacancy_categories/controller.php, blocks/vacancy_categories/css_form/repeatable-ft.form.css, blocks/vacancy_categories/css_form, blocks/vacancy_categories/db.xml, blocks/vacancy_categories/edit.php, blocks/vacancy_categories/form.php, blocks/vacancy_categories/icon.png, blocks/vacancy_categories/js_form/handlebars-helpers.js, blocks/vacancy_categories/js_form/handlebars-v4.0.4.js, blocks/vacancy_categories/js_form, blocks/vacancy_categories/view.php, blocks/vacancy_categories, blocks/video_content/add.php, blocks/video_content/composer.php, blocks/video_content/config.json, blocks/video_content/controller.php, blocks/video_content/db.xml, blocks/video_content/edit.php, blocks/video_content/form.php, blocks/video_content/icon.png, blocks/video_content/view.php, blocks/video_content, elements/blocks/page_list_plus/form.php, elements/blocks/page_list_plus/form/filters.php, elements/blocks/page_list_plus/form/loading.php, elements/blocks/page_list_plus/form/searchbox.php, elements/blocks/page_list_plus/form/sorting.php, elements/blocks/page_list_plus/form/submit.php, elements/blocks/page_list_plus/form, elements/blocks/page_list_plus/pagination.php, elements/blocks/page_list_plus/search/filters/multiple_option.php, elements/blocks/page_list_plus/search/filters, elements/blocks/page_list_plus/search, elements/blocks/page_list_plus/title.php, elements/blocks/page_list_plus/vacancy-hero-filter/form.php, elements/blocks/page_list_plus/vacancy-hero-filter/form/filters.php, elements/blocks/page_list_plus/vacancy-hero-filter/form/submit.php, elements/blocks/page_list_plus/vacancy-hero-filter/form, elements/blocks/page_list_plus/vacancy-hero-filter/search/filters/multiple_option.php, elements/blocks/page_list_plus/vacancy-hero-filter/search/filters, elements/blocks/page_list_plus/vacancy-hero-filter/search, elements/blocks/page_list_plus/vacancy-hero-filter, elements/blocks/page_list_plus, elements/blocks, src/Concrete/Flag/CustomInterfaceFlagHelper.php, src/Concrete/Flag, src/Concrete
Concrete Cache Settings
Block Cache - On
Overrides Cache - On
Full Page Caching - On - If blocks on the particular page allow it.
Full Page Cache Lifetime - Every 6 hours (default setting).
Server Software
Apache
Server API
cgi-fcgi
PHP Version
8.2.11
PHP Extensions
bcmath, bz2, calendar, cgi-fcgi, Core, ctype, curl, date, dba, dom, enchant, exif, fileinfo, filter, ftp, gd, gettext, gmp, hash, iconv, imagick, imap, intl, json, ldap, libxml, mbstring, mysqli, mysqlnd, odbc, openssl, pcntl, pcre, PDO, pdo_mysql, PDO_ODBC, pdo_pgsql, pdo_sqlite, pgsql, Phar, posix, pspell, random, readline, redis, Reflection, session, SimpleXML, soap, sockets, sodium, SPL, sqlite3, standard, sysvmsg, sysvsem, sysvshm, tidy, tokenizer, xml, xmlreader, xmlwriter, xsl, Zend OPcache, zip, zlib
PHP Settings
max_execution_time - 600
max_file_uploads - 20
max_input_nesting_level - 64
max_input_time - 600
max_input_vars - 1000
max_multipart_body_parts - -1
memory_limit - 1G
post_max_size - 256M
upload_max_filesize - 256M
zend.exception_string_param_max_len - 0
ldap.max_links - Unlimited
mbstring.regex_retry_limit - 1000000
mbstring.regex_stack_limit - 100000
mysqli.max_links - Unlimited
mysqli.max_persistent - Unlimited
odbc.max_links - Unlimited
odbc.max_persistent - Unlimited
pcre.backtrack_limit - 1000000
pcre.recursion_limit - 100000
pgsql.max_links - Unlimited
pgsql.max_persistent - Unlimited
redis.pconnect.connection_limit - 0
session.cache_limiter - no value
session.gc_maxlifetime - 7200
soap.wsdl_cache_limit - 5
unserialize_max_depth - 4096
opcache.jit_bisect_limit - 0
opcache.jit_max_exit_counters - 8192
opcache.jit_max_loop_unrolls - 8
opcache.jit_max_polymorphic_calls - 2
opcache.jit_max_recursive_calls - 2
opcache.jit_max_recursive_returns - 2
opcache.jit_max_root_traces - 1024
opcache.jit_max_side_traces - 128
opcache.max_accelerated_files - 10000
opcache.max_file_size - 0
opcache.max_wasted_percentage - 5
Anyone facing these issues?
Yes, we have this same issue when running from the Dashboard or the CLI. We have stopped using these tasks because of it.
Most of our website need these tasks. How did you fix this?
I noticed that Version 9.2.2 has just been released so maybe this has been fixed in the new version.
I read through them but there is noting about automation. Or am I missing something:
When I run these tasks locally PHP uses 90-95% of my CPU…
I am also facing similar issues. I created a batch tasks and it fails after running 1 item. I also experience issues with other Core tasks like “Remove old page versions”. No clear error, the tasks just freezes. @EnoRm did you find a solution or workaround?
I found one lead in my crontab error message that the EntityManager was closed. If that is the case i can imagine that handeling batch operations fails. Something seems to break the handling of the tasks. Closed entity manager normally indicates database errors but our hosting provider could not find any corresponding errors…
Hi @EnoRm - I would see if you can see if you can reproduce this in a Concrete demo site this or provide a link to a Github example where the problem is recreated. I looked at this with some team members in the demo sites, but the rescanning task seems to be running fine, so zeroing in on the actual task that is failing and having some resources to recreate this would be helpful.
Hi @EvanCooper, we are checking the options. Would a demo website be helpful? All the action is happening on the background. I looks like it’s a problem with all the tasks. As soon as one of them is running the CPU load is going through the roof. When they take longer (rescan files or generate thumbnails) the server just crashes.
Could you send your server settings? It might be an issue on the server side.
Hi @EnoRm - yeah if you could get a reproducible environment created and pushed up to Github and provide a link so folks could pull it down and test themselves that would be helpful to troubleshoot. 
GitHub - marcelaerts/concrete → Composer installed Concrete CMS 9 website. No custom coding.
https://test.enormblackbox.com → Test environment with Atomik Full install. Only difference (added a lot more images to the File Manager).
Running tasks with a clean install database (Atomik Full) will run correctly. As soon as I drop in more images or a lot more pages the tasks slow down and crash. I suggest you do the same to test.
Do you need more information?
We’re gonna need an admin username and password for the test.enormblackbox.com in order to take a look.