Use a custom JSON serializer

Oct 13, 2025 by magicalceylontours.com 64 views

Bug: Numbers are Converted to Scientific Notation when Generating a Local File in JSON Format

Introduction

When generating a local file in JSON format using SeaTunnel, numbers are being converted to scientific notation. This issue occurs when using JDBC as the source and LocalFile as the sink, with the JSON format specified for the output. In this article, we will explore the problem, provide a detailed analysis, and discuss possible solutions.

Problem Description

When exporting data from a database, the field fe_result with a type of DECIMAL(10,2) is being converted to scientific notation in the exported JSON file. This issue occurs when using JDBC as the source and LocalFile as the sink, with the JSON format specified for the output.

Database Table Structure

The table DDL SQL is as follows:

CREATE TABLE `data_collection` (
  `id` int(11)  NOT NULL COMMENT '主键id',
  `code` varchar(50) NOT NULL COMMENT '编号',
  `fe_result` decimal(10,2) DEFAULT NULL COMMENT '记录值',
  PRIMARY KEY (`id`) USING BTREE
) ENGINE=InnoDB COMMENT='数据';

Database Data

The data in the DB-table is as follows:

id：12312；code：1001015；fe_result：80.00

JSON Result File

The JSON file generated by SeaTunnel is as follows:

{"id":12312,"code":"1001015","fe_result":8E+1}

SeaTunnel Version

The SeaTunnel version used is 2.3.9.

SeaTunnel Config

The SeaTunnel config file is as follows:

env {
  job.mode = "BATCH"  # 离线批处理模式
}

source {
  Jdbc{
    parallelism=1
    plugin_output = "data_1"
    url = "jdbc:mysql://ip:p/data_db?useUnicode=true&characterEncoding=utf-8&serverTimezone=Asia/Shanghai"
    user = ""
    password = ""
    driver = "com.mysql.cj.jdbc.Driver"
    query = "select id, code, fe_result from data_collection"
  }

}

sink {
    LocalFile {
        plugin_input = "data_1"
        path = "../files/json"
        file_format_type = "json"
    }
}

Running Command

The running command used is as follows:

./bin/seatunnel.sh --config config/mysql2file_json_for_2.3.9.config -m local

Error Exception

There is no exception in the log file.

Zeta or Flink or Spark Version

The Zeta version used is seatunnel-2.3.9.

Java or Scala Version

The Java version used is JDK 23.

Possible Solutions

There are several possible solutions to this issue:

Use a different data type: Instead of using DECIMAL(10,2), consider using a different data type such as FLOAT or DOUBLE that can handle scientific notation.
Use a custom JSON serializer: Create a custom JSON serializer that can handle decimal numbers without converting them to scientific notation.
Use a different JSON format: Consider using a different JSON format such as JSON-LD or CBOR that can handle decimal numbers without converting them to scientific notation.
Fix the issue in SeaTunnel: If the issue is caused by a bug in SeaTunnel, consider submitting a pull request to fix the issue.

Conclusion

In conclusion, the issue of numbers being converted to scientific notation when generating a local file in JSON format using SeaTunnel is a complex problem that requires a detailed analysis. There are several possible solutions to this issue, including using a different data type, creating a custom JSON serializer, using a different JSON format, or fixing the issue in SeaTunnel. By understanding the root cause of the issue and exploring possible solutions, we can resolve this problem and ensure that our data is exported correctly.
Q&A: Bug - Numbers are Converted to Scientific Notation when Generating a Local File in JSON Format

Introduction

In our previous article, we discussed the issue of numbers being converted to scientific notation when generating a local file in JSON format using SeaTunnel. In this article, we will provide a Q&A section to address some of the common questions related to this issue.

Q: What is the root cause of this issue?

A: The root cause of this issue is the way SeaTunnel handles decimal numbers when generating a JSON file. When a decimal number is converted to a string, it is being converted to scientific notation, which is not the expected behavior.

Q: Why is this issue only occurring with decimal numbers?

A: This issue is only occurring with decimal numbers because of the way they are being handled by SeaTunnel. When a decimal number is converted to a string, it is being converted to scientific notation, which is not the expected behavior.

Q: Can this issue be fixed by changing the data type of the field?

A: Yes, this issue can be fixed by changing the data type of the field to a type that can handle decimal numbers without converting them to scientific notation. For example, you can change the data type of the field to FLOAT or DOUBLE.

Q: Can this issue be fixed by creating a custom JSON serializer?

A: Yes, this issue can be fixed by creating a custom JSON serializer that can handle decimal numbers without converting them to scientific notation.

Q: Can this issue be fixed by using a different JSON format?

A: Yes, this issue can be fixed by using a different JSON format such as JSON-LD or CBOR that can handle decimal numbers without converting them to scientific notation.

Q: Is this issue specific to SeaTunnel or can it occur in other tools as well?

A: This issue can occur in other tools as well, but it is more likely to occur in tools that use a similar JSON serialization mechanism as SeaTunnel.

Q: How can I prevent this issue from occurring in the future?

A: To prevent this issue from occurring in the future, you can use a different data type for the field, create a custom JSON serializer, use a different JSON format, or fix the issue in SeaTunnel.

Q: Can I submit a pull request to fix this issue in SeaTunnel?

A: Yes, you can submit a pull request to fix this issue in SeaTunnel. However, please make sure to follow the contribution guidelines and ensure that your pull request is well-documented and tested.

Q: How can I get help if I am experiencing this issue?

A: If you are experiencing this issue, you can get help by reaching out to the SeaTunnel community, posting on the SeaTunnel forum, or submitting a bug report on the SeaTunnel issue tracker.

Conclusion

In conclusion, the issue of numbers being converted to scientific notation when generating a local file in JSON format using SeaTunnel is a complex problem that requires a detailed analysis. By understanding the root cause of the issue and exploring possible solutions, we can resolve this problem and ensure that our data is exported correctly.