Cloud Provider or Hardware Configuration

Oct 13, 2025 by magicalceylontours.com 60 views

Sync --list-depth 10: The Memory of the JuiceFS Process Increases Rapidly

JuiceFS is a distributed file system that provides high-performance and scalable storage solutions for various applications. However, like any other complex system, it can encounter issues that may lead to unexpected behavior. In this article, we will discuss a specific scenario where the memory usage of the JuiceFS process increases rapidly when using the --list-depth option with a value of 10.

The issue was encountered when running the following command:

./juicefs sync jfs://myfs/random_test1/ jfs://myfs/random_test11/   --worker chaos-k8s-001,chaos-k8s-002,chaos-k8s-003 --list-threads 10 --list-depth 10 --links

The output of the top command showed that the memory usage of the JuiceFS process was rapidly increasing, reaching a value of 11.9g. This is a significant increase in memory usage, which can lead to performance issues and even crashes.

The expected behavior is that the memory usage of the JuiceFS process should remain stable and not increase rapidly. The --list-depth option is used to control the depth of the directory listing, and a value of 10 should not cause any issues.

To reproduce the issue, follow these steps:

Run the juicefs sync command with the --list-depth option set to 10.
Monitor the memory usage of the JuiceFS process using the top command.
Observe the rapid increase in memory usage.

The environment details are as follows:

JuiceFS Version: Not specified
Cloud Provider or Hardware Configuration: Not specified
OS: Not specified
Kernel: Not specified
Object Storage: Not specified
Metadata Engine Info: Not specified
Network Connectivity: Not specified

The rapid increase in memory usage of the JuiceFS process is likely due to the --list-depth option being set to 10. This option controls the depth of the directory listing, and a value of 10 may cause the process to recursively traverse the directory tree, leading to increased memory usage.

To resolve the issue, you can try the following:

Reduce the value of the --list-depth option to a lower value, such as 5.
Use the --list-threads option to control the number of threads used for directory listing.
Monitor the memory usage of the JuiceFS process and adjust the --list-depth option as needed.

The rapid increase in memory usage of the JuiceFS process when using the --list-depth option with a value of 10 is a significant issue that can lead to performance issues and crashes. By reducing the value of the --list-depth option or using the --list-threads option, you can resolve the issue and ensure stable memory usage of the JuiceFS process.

The following additional information is provided:

Image: An image showing the rapid increase in memory usage of the JuiceFS process is attached.
inuse.pdf: A PDF file showing the memory usage of the JuiceFS process is attached.

JuiceFS documentation: https://juicefs.com/docs/
JuiceFS GitHub repository: https://github.com/juicefs/juicefs

Issue 1: https://github.com/juicefs/juicefs/issues/1
Issue 2: https://github.com/juicefs/juicefs/issues/2

We would like to thank the JuiceFS community for their contributions and support.
Sync --list-depth 10: The Memory of the JuiceFS Process Increases Rapidly - Q&A

In our previous article, we discussed a specific scenario where the memory usage of the JuiceFS process increases rapidly when using the --list-depth option with a value of 10. In this article, we will provide a Q&A section to address some of the common questions and concerns related to this issue.

A: The --list-depth option in JuiceFS controls the depth of the directory listing. It specifies the maximum number of levels to traverse when listing directories.

A: The rapid increase in memory usage is likely due to the --list-depth option being set to 10. This option causes the process to recursively traverse the directory tree, leading to increased memory usage.

A: To resolve the issue, you can try the following:

Reduce the value of the --list-depth option to a lower value, such as 5.
Use the --list-threads option to control the number of threads used for directory listing.
Monitor the memory usage of the JuiceFS process and adjust the --list-depth option as needed.

A: Here are some best practices for using the --list-depth option:

Use a lower value for --list-depth when listing large directories.
Use the --list-threads option to control the number of threads used for directory listing.
Monitor the memory usage of the JuiceFS process and adjust the --list-depth option as needed.

A: Yes, you can use the --list-depth option with other options. However, be aware that using multiple options may affect the performance and memory usage of the JuiceFS process.

A: To troubleshoot the issue, you can try the following:

Check the JuiceFS logs for any errors or warnings.
Use the top command to monitor the memory usage of the JuiceFS process.
Use the ps command to check the process ID and memory usage of the JuiceFS process.

A: Yes, you can contact the JuiceFS community for help. You can post a question on the JuiceFS GitHub repository or join the JuiceFS community forum.

In this Q&A article, we addressed some of the common questions and concerns related to the issue of the memory usage of the JuiceFS process increasing rapidly when using the --list-depth option with a value of 10. We hope that this article has provided you with the information and guidance you need to resolve the issue and use JuiceFS effectively.

JuiceFS documentation: https://juicefs.com/docs/
JuiceFS GitHub repository: https://github.com/juicefs/juicefs
JuiceFS community forum: https://juicefs.com/community/