Repository navigation
[Bug]: Upgrade 30.0.10 to 31.04 (dav) requires 6GB of RAM on single-user instance #52505
Description
⚠️ This issue respects the following points: ⚠️
- This is a bug, not a question or a configuration/webserver/proxy issue.
- This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
- Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
- I agree to follow Nextcloud's Code of Conduct.
Bug description
When trying to upgrade 30.0.10 to 31.0.4, the occ upgrade command exits without error after trying to update the dav app. Increasing memory_limit to ludicrous amounts of memory seems to help.
Steps to reproduce
- Have 30.0.10 running
- Update nextcloud using the web updater
- run
php occ upgradeon the shell - php process gets killed, message in log says memory limit exceeded
Expected behavior
occ upgrade runs successfully and does not allocate multiple gigabytes of memory
Nextcloud Server version
31
Operating system
Debian/Ubuntu
PHP engine version
PHP 8.2
Web server
Nginx
Database engine version
PostgreSQL
Is this bug present after an update or on a fresh install?
Upgraded to a MAJOR version (ex. 31 to 32)
Are you using the Nextcloud Server Encryption module?
Encryption is Disabled
What user-backends are you using?
- Default user-backend (database)
- LDAP/ Active Directory
- SSO - SAML
- Other
Configuration report
{
"system": {
"instanceid": "***REMOVED SENSITIVE VALUE***",
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": [
"warnochwas.de"
],
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"dbtype": "pgsql",
"version": "30.0.10.1",
"overwrite.cli.url": "https:\/\/warnochwas.de",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbport": "",
"dbtableprefix": "oc_",
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"mail_from_address": "***REMOVED SENSITIVE VALUE***",
"mail_smtpmode": "sendmail",
"mail_sendmailmode": "smtp",
"mail_domain": "***REMOVED SENSITIVE VALUE***",
"mail_smtphost": "***REMOVED SENSITIVE VALUE***",
"mail_smtpport": "25",
"memcache.locking": "\\OC\\Memcache\\Redis",
"memcache.distributed": "\\OC\\Memcache\\Redis",
"memcache.local": "\\OC\\Memcache\\Redis",
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"port": 6379,
"dbindex": 7
},
"default_phone_region": "DE",
"maintenance": true,
"theme": "",
"loglevel": 0,
"maintenance_window_start": 0,
"app_install_overwrite": [
"maps",
"diary"
],
"updater.secret": "***REMOVED SENSITIVE VALUE***"
}
}List of activated Apps
- federation: 1.21.0
- files: 2.3.1
- files_downloadlimit: 3.0.0
- files_external: 1.22.0
- files_pdfviewer: 3.0.0
- files_reminders: 1.3.0
- files_sharing: 1.22.0
- files_trashbin: 1.20.1
- files_versions: 1.23.0
- firstrunwizard: 3.0.0
- logreader: 3.0.0
- lookup_server_connector: 1.19.0
- nextcloud_announcements: 2.0.0
- notifications: 3.0.0
- oauth2: 1.19.1
- password_policy: 3.0.0
- photos: 4.0.0-dev.1
- polls: 7.4.2
- privacy: 2.0.0
- profile: true
- provisioning_api: 1.20.0
- recommendations: 3.0.0
- related_resources: 1.5.0
- serverinfo: 2.0.0
- settings: 1.13.0
- sharebymail: 1.20.0
- socialsharing_email: 3.3.0
- support: 2.0.0
- survey_client: 2.0.0
- systemtags: 1.20.0
- text: 4.1.0
- theming: 2.6.0
- twofactor_backupcodes: 1.19.0
- twofactor_nextcloud_notification: 4.0.0
- twofactor_totp: 12.0.0-dev
- twofactor_webauthn: 2.1.0
- updatenotification: 1.20.0
- user_status: 1.10.0
- viewer: 3.0.0
- weather_status: 1.10.0
- webhook_listeners: 1.1.0-dev
- workflowengine: 2.12.0
Disabled:
- admin_audit: 1.21.0
- deck: 1.14.4 (installed 1.14.4)
- diary: 0.0.11 (installed 0.0.11)
- encryption: 2.19.0
- files_retention: 1.19.1 (installed 1.19.1)
- files_rightclick: 0.15.1 (installed 1.6.0)
- maps: 1.5.0 (installed 1.5.0)
- onlyoffice: 9.8.0 (installed 9.8.0)
- suspicious_login: 9.0.1
- user_ldap: 1.22.0Nextcloud Signing status
Nextcloud Logs
[carlene][root@/var/www/nextcloud] > sudo -u www-data php occ upgrade -vvv
The process control (PCNTL) extensions are required in case you want to interrupt long running commands - see https://www.php.net/manual/en/book.pcntl.php
Additionally the function 'pcntl_signal' and 'pcntl_signal_dispatch' need to be enabled in your php.ini.
Nextcloud or one of the apps require upgrade - only a limited number of commands are available
You may use your browser or the occ upgrade command to do the upgrade
2025-04-28T13:52:44+00:00 Setting log level to debug
2025-04-28T13:52:44+00:00 Turned on maintenance mode
2025-04-28T13:52:44+00:00 Repair step: Repair MySQL collation
2025-04-28T13:52:44+00:00 Repair info: Not a mysql database -> nothing to do
2025-04-28T13:52:44+00:00 Repair step: Copy data from accounts table when migrating from ownCloud
2025-04-28T13:52:44+00:00 Repair step: Drop account terms table when migrating from ownCloud
2025-04-28T13:52:44+00:00 Updating database schema
2025-04-28T13:52:44+00:00 Updated database
2025-04-28T13:52:44+00:00 Updating <dav> ...
2025-04-28T13:52:44+00:00 Repair step: Fix component of birthday calendars
2025-04-28T13:52:44+00:00 Repair info: 2 birthday calendars updated.
2025-04-28T13:52:44+00:00 Repair step: Regenerating birthday calendars to use new icons and fix old birthday events without year
2025-04-28T13:52:44+00:00 Repair info: Repair step already executed
2025-04-28T13:52:44+00:00 Repair step: Fix broken values of calendar objects
2025-04-28T13:52:44+00:00 Fix broken values of calendar objects
2025-04-28T13:52:44+00:00
2025-04-28T13:52:44+00:00 Done
0/0 [->--------------------------] 0%2025-04-28T13:52:44+00:00
2025-04-28T13:52:44+00:00 Repair step: Registering building of calendar search index as background job
2025-04-28T13:52:44+00:00 Repair info: Repair step already executed
2025-04-28T13:52:44+00:00 Repair step: Register building of social profile search index as background job
2025-04-28T13:52:44+00:00 Repair info: Repair step already executed
2025-04-28T13:52:44+00:00 Repair step: Registering background jobs to update cache for webcal calendars
2025-04-28T13:52:44+00:00 Repair info: Added 0 background jobs to update webcal calendars
2025-04-28T13:52:44+00:00 Repair step: Registering building of calendar reminder index as background job
2025-04-28T13:52:44+00:00 Repair info: Repair step already executed
2025-04-28T13:52:44+00:00 Repair step: Clean up orphan event and contact data
2025-04-28T13:52:44+00:00 Repair info: 0 events without a calendar have been cleaned upAdditional info
I have temporarily increased phps memory_limit to 8192M (8GB) - that seems to work, however my default of 512MB or even 2048MB were not enough for a successful upgrade.
Offending, long-running SQL query seems to be (caution, line is cut off):
DELETE FROM "oc_calendarobjects_props" WHERE "id" IN ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28, $29, $30, $31, $32, $33, $34, $35, $36, $37, $38, $39, $40, $41, $42, $43, $44, $45, $46, $47, $48, $49, $50, $51, $52, $53, $54, $55, $56, $57, $58, $59, $60, $61, $62, $63, $64, $65, $66, $67, $68, $69, $70, $71, $72, $73, $74, $75, $76, $77, $78, $79, $80, $81, $82, $83, $84, $85, $86, $87, $88, $89, $90, $91, $92, $93, $94, $95, $96, $97, $98, $99, $100, $101, $102, $103, $104, $105, $106, $107, $108, $109, $110, $111, $112, $113, $114, $115, $116, $117, $118, $119, $120, $121, $122, $123, $124, $125, $126, $127, $128, $129, $130, $131, $132, $133, $134, $135, $136, $137, $138, $139, $140, $141, $142, $143, $144, $145, $146, $147, $148, $149, $150, $151, $152, $153, $154, $155, $156, $157, $158, $159, $160, $161, $162, $163, $164, $165, $166, $167, $168, $169, $170, $171, $172, $173, $174, $175, $176, $177, $178, $179, $18
and/or a parallel running VACUUM (i'm not sure whether the updater initiates that, my server should not automatically do that)
During the upgrade the php occ upgrade process consumed about 6GB of RAM, while https://github.com/syncloud/nextcloud/blob/master/config/php.ini#L393 recommends 1GB.
/var/www/nextcloud is about 114GB in size, nextcloud database about 13GB prior to starting the upgrade.
Metadata
Metadata
Assignees
Type
Projects
- Status
- Status
Activity
Kunsi commentedon Apr 28, 2025
Note: As of writing this bug report, the task is still ongoing. I will update the original report as soon as i get more information.
Kunsi commentedon Apr 28, 2025
Update: Yeah, after giving it 8 Gigabytes of Memory, the update went through successfully. It took 36 Minutes until the next log message appeared (note the timestamps):
Everything else took <2 minutes to execute.
Kunsi commentedon Apr 28, 2025
Full updater log:
susnux commentedon Apr 29, 2025
cc @ChristophWurst this seems to be CalDAV related.
kesselb commentedon Apr 29, 2025
Background jobRepair step causing the queries: https://github.com/nextcloud/server/blob/9df998e710d19aa78a61f8f1cad054baf2b7263a/apps/dav/lib/Migration/RemoveOrphanEventsAndContacts.phpThe
background jobrepair step is there for a while. Is it possible that recently, many events or calendars were deleted?Kunsi commentedon Apr 29, 2025
In December i removed a large ICS subscription, but since then i did multiple upgrades of my nextcloud instance which did all only ever take a couple of minutes.
susnux commentedon Apr 29, 2025
Maybe this background job can be chunked to reduce memory footprint?
We do this in other places as well.
kesselb commentedon Apr 29, 2025
Sorry, I wrote background job but meant repair step 🙈
The repair step is about removing orphaned data. What are your thoughts on marking it as expensive to skip it on upgrades?
Deletion is done in chunks of 200 ids. I think we could bump that to 1000 like we do in other places to have less expensive queries. Wrapping everything in a transaction could also help.
cc @SebastianKrupinski @st3iny
st3iny commentedon Apr 29, 2025
Yeah, we can increase the chunk size.
susnux commentedon Apr 29, 2025
I meant like with other repair steps like this:
See also
InitBackgroundImagesMigrationof thethemingappChristophWurst commentedon Apr 30, 2025
That works until you have an installation where the number of orphans created is higher than the orphans cleaned up in a given time span ;-)
I agree that this clean-up should be moved to a background job, though