Ошибка при сборке чанков код ошибки 504 nextcloud

Nextcloud community

Loading

Источник

⚠️ This issue respects the following points: ⚠️

  • This is a bug, not a question or a configuration/webserver/proxy issue.
  • This issue is not already reported on Github (I’ve searched it).
  • Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
  • I agree to follow Nextcloud’s Code of Conduct.

Bug description

Upload a file, say 400 MB. After upload progress bar completes, there is a Processing files step. After a minute or so, there is this error message Error when assembling chunks, status code 504 . Eventually, the file is uploaded and would show up in the web interface, yet, there is this error. Depending on the size of the file, clamscan takes 100% of the CPU.
Doesnt occur with smaller files say few MBs.

Nextcloud error

Steps to reproduce

  1. Upload a file, say 400 MB and wait the upload progress bar completes,
  2. Wait for the Processing files step
  3. After a minute or so, there is this error message Error when assembling chunks, status code 504
  4. Although the error message, the file is uploaded eventually after several minutes, depending on the file size. During this period all CPU time is taken by clamav

Expected behavior

No error message.

Installation method

Official All-in-One appliance

Operating system

Debian/Ubuntu

PHP engine version

PHP 7.4

Web server

Apache (supported)

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install?

Updated to a major version (ex. 22.2.3 to 23.0.1)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO — SAML
  • Other

Configuration report

## Server configuration detail

**Operating system:** Linux 5.4.0-107-generic #121-Ubuntu SMP Thu Mar 24 16:04:27 UTC 2022 x86_64

**Webserver:** Apache (fpm-fcgi)

**Database:** pgsql PostgreSQL 12.9 (Ubuntu 12.9-0ubuntu0.20.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, 64-bit

**PHP version:** 7.4.3

Modules loaded: Core, date, libxml, openssl, pcre, zlib, filter, hash, Reflection, SPL, session, standard, sodium, cgi-fcgi, PDO, xml, apcu, bcmath, bz2, calendar, ctype, curl, dom, mbstring, FFI, fileinfo, ftp, gd, gettext, gmp, iconv, imagick, imap, intl, json, ldap, exif, pdo_pgsql, pgsql, apc, posix, readline, shmop, SimpleXML, soap, sockets, sysvmsg, sysvsem, sysvshm, tokenizer, xmlreader, xmlwriter, xsl, zip, Phar, Zend OPcache

**Nextcloud version:** 23.0.3 - 23.0.3.2

**Updated from an older Nextcloud/ownCloud or fresh install:** 

**Where did you install Nextcloud from:** unknown

<details><summary>Signing status</summary>

Array
(
)

</details>

<details><summary>List of activated apps</summary>


Enabled:
 - accessibility: 1.9.0
 - activity: 2.15.0
 - admin_audit: 1.13.0
 - afterlogic: 2.0.3
 - analytics: 4.2.1
 - announcementcenter: 6.1.1
 - appointments: 1.12.2
 - apporder: 0.15.0
 - calendar: 3.2.2
 - checksum: 1.1.3
 - circles: 23.1.0
 - cloud_federation_api: 1.6.0
 - cms_pico: 1.0.18
 - comments: 1.13.0
 - contacts: 4.1.0
 - contactsinteraction: 1.4.0
 - dashboard: 7.3.0
 - dav: 1.21.0
 - deck: 1.6.1
 - event_update_notification: 1.5.0
 - external: 3.10.2
 - federatedfilesharing: 1.13.0
 - federation: 1.13.0
 - files: 1.18.0
 - files_antivirus: 3.2.2
 - files_external: 1.15.0
 - files_pdfviewer: 2.4.0
 - files_rightclick: 1.2.0
 - files_sharing: 1.15.0
 - files_trashbin: 1.13.0
 - files_versions: 1.16.0
 - files_videoplayer: 1.12.0
 - firstrunwizard: 2.12.0
 - forms: 2.5.0
 - fulltextsearch_elasticsearch: 23.0.0
 - groupfolders: 11.1.2
 - integration_onedrive: 1.1.2
 - logreader: 2.8.0
 - lookup_server_connector: 1.11.0
 - maps: 0.1.10
 - music: 1.5.1
 - nextcloud_announcements: 1.12.0
 - notes: 4.3.1
 - notifications: 2.11.1
 - oauth2: 1.11.0
 - onlyoffice: 7.3.2
 - password_policy: 1.13.0
 - photos: 1.5.0
 - privacy: 1.7.0
 - provisioning_api: 1.13.0
 - ransomware_protection: 1.12.0
 - recommendations: 1.2.0
 - serverinfo: 1.13.0
 - settings: 1.5.0
 - sharebymail: 1.13.0
 - support: 1.6.0
 - survey_client: 1.11.0
 - systemtags: 1.13.0
 - tasks: 0.14.4
 - text: 3.4.1
 - theming: 1.14.0
 - twofactor_backupcodes: 1.12.0
 - twofactor_totp: 6.2.0
 - unsplash: 1.2.4
 - updatenotification: 1.13.0
 - user_retention: 1.6.0
 - user_status: 1.3.1
 - viewer: 1.7.0
 - weather_status: 1.3.0
 - workflow_script: 1.8.0
 - workflowengine: 2.5.0
Disabled:
 - encryption
 - user_ldap

Configuration (config/config.php)

{
    "passwordsalt": "***REMOVED SENSITIVE VALUE***",
    "secret": "***REMOVED SENSITIVE VALUE***",
    "trusted_domains": [
        "localhost",
        "*.*.*.*",
        "nextcloud",
        "www.cloudcreator.eu"
    ],
    "datadirectory": "***REMOVED SENSITIVE VALUE***",
    "dbtype": "pgsql",
    "version": "23.0.3.2",
    "overwrite.cli.url": "https://nextcloud/",
    "dbname": "***REMOVED SENSITIVE VALUE***",
    "dbhost": "***REMOVED SENSITIVE VALUE***",
    "dbport": "",
    "dbtableprefix": "oc_",
    "dbuser": "***REMOVED SENSITIVE VALUE***",
    "dbpassword": "***REMOVED SENSITIVE VALUE***",
    "installed": true,
    "instanceid": "***REMOVED SENSITIVE VALUE***",
    "log_type": "file",
    "logfile": "/var/log/nextcloud/nextcloud.log",
    "loglevel": "2",
    "log.condition": {
        "apps": [
            "admin_audit"
        ]
    },
    "mail_smtpmode": "smtp",
    "logtimezone": "Europe/Brussels",
    "maintenance": false,
    "updater.release.channel": "stable",
    "memcache.local": "\OC\Memcache\APCu",
    "default_phone_region": "BE",
    "mail_domain": "***REMOVED SENSITIVE VALUE***",
    "mail_from_address": "***REMOVED SENSITIVE VALUE***",
    "mail_sendmailmode": "smtp",
    "mail_smtpauthtype": "LOGIN",
    "mail_smtpauth": 1,
    "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
    "mail_smtpport": "25",
    "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
    "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
    "theme": ""
}

Cron Configuration: Array
(
[backgroundjobs_mode] => cron
[lastcron] => 1649751002
)

External storages: yes

External storage configuration

Encryption: no

User-backends:

  • OCUserDatabase

Browser: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:99.0) Gecko/20100101 Firefox/99.0



### List of activated Apps

```shell
<details><summary>List of activated apps</summary>


Enabled:
 - accessibility: 1.9.0
 - activity: 2.15.0
 - admin_audit: 1.13.0
 - afterlogic: 2.0.3
 - analytics: 4.2.1
 - announcementcenter: 6.1.1
 - appointments: 1.12.2
 - apporder: 0.15.0
 - calendar: 3.2.2
 - checksum: 1.1.3
 - circles: 23.1.0
 - cloud_federation_api: 1.6.0
 - cms_pico: 1.0.18
 - comments: 1.13.0
 - contacts: 4.1.0
 - contactsinteraction: 1.4.0
 - dashboard: 7.3.0
 - dav: 1.21.0
 - deck: 1.6.1
 - event_update_notification: 1.5.0
 - external: 3.10.2
 - federatedfilesharing: 1.13.0
 - federation: 1.13.0
 - files: 1.18.0
 - files_antivirus: 3.2.2
 - files_external: 1.15.0
 - files_pdfviewer: 2.4.0
 - files_rightclick: 1.2.0
 - files_sharing: 1.15.0
 - files_trashbin: 1.13.0
 - files_versions: 1.16.0
 - files_videoplayer: 1.12.0
 - firstrunwizard: 2.12.0
 - forms: 2.5.0
 - fulltextsearch_elasticsearch: 23.0.0
 - groupfolders: 11.1.2
 - integration_onedrive: 1.1.2
 - logreader: 2.8.0
 - lookup_server_connector: 1.11.0
 - maps: 0.1.10
 - music: 1.5.1
 - nextcloud_announcements: 1.12.0
 - notes: 4.3.1
 - notifications: 2.11.1
 - oauth2: 1.11.0
 - onlyoffice: 7.3.2
 - password_policy: 1.13.0
 - photos: 1.5.0
 - privacy: 1.7.0
 - provisioning_api: 1.13.0
 - ransomware_protection: 1.12.0
 - recommendations: 1.2.0
 - serverinfo: 1.13.0
 - settings: 1.5.0
 - sharebymail: 1.13.0
 - support: 1.6.0
 - survey_client: 1.11.0
 - systemtags: 1.13.0
 - tasks: 0.14.4
 - text: 3.4.1
 - theming: 1.14.0
 - twofactor_backupcodes: 1.12.0
 - twofactor_totp: 6.2.0
 - unsplash: 1.2.4
 - updatenotification: 1.13.0
 - user_retention: 1.6.0
 - user_status: 1.3.1
 - viewer: 1.7.0
 - weather_status: 1.3.0
 - workflow_script: 1.8.0
 - workflowengine: 2.5.0
Disabled:
 - encryption
 - user_ldap

«`

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

https://www.cloudcreator.eu/index.php/s/4yj4bQTiHqqdntB

Additional info

No response

Источник

Thanks for changing the AIO documentation @szaimen. It is true that even nextcloud’s basic browser uploader in standard configuration splits large files into 10MB chunks, except for the public file drop situation.

This is great because the data transfer part of an upload is very unlikely to time out then. A client could even be smart enough that if a single chunk transfer times out it would be retried with a smaller chunk size, which I believe the desktop client already does.

Now once the chunks are transferred to nextcloud, they are re-assembled into a single file. Depending on the storage used as well as the number and size of the chunks, this process can take quite some time. The re-assembling must be finished within the timeout set for a single HTTP request. Probably since nextcloud doesn’t send any data to a client during a lengthy re-assembly process, proxies and/or clients are likely to discard the connection, typically to protect from (unintentional) DOS. Discarding the connection will fail the upload. The time for re-assembly can increase massively if nextcloud is configured to use networked storage (like iSCSI, NFS, SFTP, S3) and moves chunks and final file in between remote storage and local temp storage several times.

As a user, currently I have two choices to transfer a large file:

  1. Use a client that supports chunks (nextcloud browser interface or desktop client), but that will time out on the re-assembly part, after which even a successful upload is discarded.
  2. Use a client that doesn’t use chunked uploads (such as rclone or a WebDAV client), which will time out on the actual data transfer part.

Changing timeouts and chunk sizes in configuration can mitigate the problem under certain conditions, while degrading security and performance in some cases, but it is not a reliable solution. Ideally clients, and in particular the desktop client, would be able to wait for the re-assembling of chunks and register a successful upload.

Currently this doesn’t seem to be implemented, or is not working. This is a problem for users that have to deal with large files. It will hardly affect users that deal with typical office documents, but those working with audio, video, large documents for print, disk images, etc, will have a very hard time.

I’m one of the folks who has their nextcloud configuration cranked up to 11 regarding timeouts, request body sizes, etc, and still am unable to get my 8 GB digital video file uploaded, while I have no issues to, for instance, upload it to my peertube instance which is self-hosted just the same.

When exchanging large files with collaborators I would love to avoid having to use WeTransfer or DropBox or other proprietary and centralized tools, but sometimes I have to rely on them, because these services figured out how to deal with large files reliably. It would also be amazing for people currently using CreativeCloud to move to sovereign storage with nextcloud. But as things stand I would only be able to recommend nextcloud for collaborating on light office files.

Источник


    • #1

    Hi Everyone,

    I am facing this issue when I’m trying to upload bigger files to my NextCloud. I definitely need to have the possibility to upload bigger files as I need to share files for video editing — it’s not uncommon to have 20GB+ files. How can I solve this issue? I saw some forum posts where they are talking about an apache issue but that shouldn’t be relevant to me. Not sure what information you need so I attached some screenshots to have some info about the system.

    1. Running containers in Portainer

    2. Basic system info (CPU, Kernel, etc.)

    3. File systems

    4. Disks

    • Offizieller Beitrag
    • #2
    • #3

    Thanks for the links, I checked them out but I’m not sure what to do with it to be honest. It seems like I will have some help during the weekend and hopefully he is going to help me figuring out what to do. I suspect I have some issues with encryption. I basically just want to turn it off but unfortunately it’s not as easy as it is to turn it on.

    • #4

    Maybe look at «step 9» at https://www.linuxbabe.com/ubun…tu-20-04-nginx-lemp-stack. Modify the size according to your needs. Maybe the files are in different locations, but if you look at the instructions above of the guide, you should be able to understand what should be in the files (i.e. how they should look like to know if you are working on the correct file).

Источник

Had to add this to nginx proxy config file

location / {
        proxy_headers_hash_max_size 512;
        proxy_headers_hash_bucket_size 64;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        add_header Front-End-Https on;
        proxy_pass http(s)://<your NextCloud ip and port>;
    }

Also add client_max_body 10G; to NextCloud nginx.conf and post_max_size = 10G in php.ini

answered Mar 17, 2020 at 16:14

JordanPlayz158's user avatar

1

Источник

  • Ошибка при регистрации на закупки гов
  • Ошибка при сборке visual studio
  • Ошибка при сборке ресурсов win32 формат потока значков отличается от ожидаемого
  • Ошибка при регистрации на госуслугах что делать
  • Ошибка при сборе данных для административные шаблоны