Jackson - Installation

本文介绍如何在Java应用中安装和配置Jackson JSON API。Jackson由三个核心项目组成:Jackson Core、Jackson Annotations 和 Jackson Databind。文中详细说明了通过Maven依赖和直接添加JAR文件两种方式来集成Jackson。

The Java JSON API called Jackson consists of one core JAR file (project) and two other JAR files that use the core JAR file. The three JAR files (projects) in the Jackson JSON API are:

  • Jackson Core
  • Jackson Annotations
  • Jackson Databind

These projects use each other in that sequence too. Jackson Annotation uses the Jackson Core features, and the Jackson Databind uses Jackson Annotation.

In order to "install" Jackson in your Java application you need to add these JAR files to the classpath of your application. There are several ways to do so. I will cover two here.

Adding the JAR Files to Your Classpath

In order to use any of the Jackson projects (JAR files) you must add the corresponding JAR file to the classpath of your application. You can do so in several different ways.

Before you even start adding the Jackson JAR files to your classpath, you need to decide what version of the Jackson JSON API to use. To do so, go to the Jackson website and see what versions are available. If you are not already using a specific version of Jackson in your project, use the latest stable version. Write the version number down. You may need it later.

You can find a list of Jackson releases here:

https://github.com/FasterXML/jackson-core/releases

Adding Jackson as Maven Dependencies

If your project uses Maven to build (and resolve + download dependencies), you need to add Jackson as a dependency to your project's POM file. Here is how Jackson as Maven dependencies look:

<dependency>
  <groupId>com.fasterxml.jackson.core</groupId>
  <artifactId>jackson-core</artifactId>
  <version>2.5.3</version>
</dependency>

<dependency>
  <groupId>com.fasterxml.jackson.core</groupId>
  <artifactId>jackson-annotations</artifactId>
  <version>2.5.3</version>
</dependency>

<dependency>
  <groupId>com.fasterxml.jackson.core</groupId>
  <artifactId>jackson-databind</artifactId>
  <version>2.5.3</version>
</dependency>

Note, that both jackson-annotations and jackson-databind have "transitive dependencies" to jackson-core(and jackson-databind to jackson-annotations). This means, that if you plan to use the jackson-databindproject, you just need to include that as dependency in your Maven POM file. Then it will transitively include the other two projects as dependencies. Regardless, I like to add my dependencies explicitly so I can see what I am using (but that's just me).

Adding the JAR Files Directly to the Classpath

Another option is to add the Jackson JAR files directly to the classpath of your application. To do so you must first download the JAR files. You can download the finished JAR files via the central Maven repository. Go to:

http://search.maven.org

Search for the project you want to use (e.g. jackson-corejackson-annotation or jackson-databind). Once you find the right project, make sure you click the "all versions" link next to the version number in the search result. That will show you a list of the previous versions of that project. Click around until you find a link directly to the JAR file for that project and that version. Then download the JAR file.

Once the JAR files are downloaded, add them to the classpath of your project. You can do so within your IDE, and / or within the scripts (bash or cmd) you use to run your application.

sqoop import \ > --connect jdbc:mysql://localhost:3306/mydatabase \ > --username root \ > -P \ > --table products \ > --target-dir /user/hadoop/products \ > --fields-terminated-by ',' \ > --lines-terminated-by '\n' \ > --delete-target-dir \ > --num-mappers 1 Warning: /home/ljm1/daolun/servers/sqoop-1.4.6/../hcatalog does not exist! HCatalog jobs will fail. Please set $HCAT_HOME to the root of your HCatalog installation. Warning: /home/ljm1/daolun/servers/sqoop-1.4.6/../accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 2025-07-05 01:36:29,069 INFO [main] sqoop.Sqoop (Sqoop.java:<init>(92)) - Running Sqoop version: 1.4.6 Enter password: 2025-07-05 01:36:35,176 INFO [main] manager.MySQLManager (MySQLManager.java:initOptionDefaults(69)) - Preparing to use a MySQL streaming resultset. 2025-07-05 01:36:35,177 INFO [main] tool.CodeGenTool (CodeGenTool.java:generateORM(92)) - Beginning code generation 2025-07-05 01:36:35,718 INFO [main] manager.SqlManager (SqlManager.java:execute(757)) - Executing SQL statement: SELECT t.* FROM `products` AS t LIMIT 1 2025-07-05 01:36:35,822 INFO [main] manager.SqlManager (SqlManager.java:execute(757)) - Executing SQL statement: SELECT t.* FROM `products` AS t LIMIT 1 2025-07-05 01:36:35,850 INFO [main] orm.CompilationManager (CompilationManager.java:findHadoopJars(94)) - HADOOP_MAPRED_HOME is /home/ljm1/daolun/servers/hadoop-2.7.4 Note: /tmp/sqoop-ljm1/compile/280c35d4ceb0b5eb13ccf7a0afe259ef/products.java uses or overrides a deprecated API. Note: Recompile with -Xlint:deprecation for details. 2025-07-05 01:36:42,374 INFO [main] orm.CompilationManager (CompilationManager.java:jar(330)) - Writing jar file: /tmp/sqoop-ljm1/compile/280c35d4ceb0b5eb13ccf7a0afe259ef/products.jar SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/ljm1/daolun/servers/hadoop-2.7.4/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/ljm1/daolun/servers/hbase-1.4.0/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 2025-07-05 01:36:43,216 INFO [main] tool.ImportTool (ImportTool.java:deleteTargetDir(534)) - Destination directory /user/hadoop/products is not present, hence not deleting. 2025-07-05 01:36:43,216 WARN [main] manager.MySQLManager (MySQLManager.java:importTable(107)) - It looks like you are importing from mysql. 2025-07-05 01:36:43,217 WARN [main] manager.MySQLManager (MySQLManager.java:importTable(108)) - This transfer can be faster! Use the --direct 2025-07-05 01:36:43,217 WARN [main] manager.MySQLManager (MySQLManager.java:importTable(109)) - option to exercise a MySQL-specific fast path. 2025-07-05 01:36:43,217 INFO [main] manager.MySQLManager (MySQLManager.java:checkDateTimeBehavior(189)) - Setting zero DATETIME behavior to convertToNull (mysql) 2025-07-05 01:36:43,314 INFO [main] mapreduce.ImportJobBase (ImportJobBase.java:runImport(235)) - Beginning import of products 2025-07-05 01:36:43,375 INFO [main] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1173)) - mapred.jar is deprecated. Instead, use mapreduce.job.jar 2025-07-05 01:36:43,412 INFO [main] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1173)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address 2025-07-05 01:36:43,458 INFO [main] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1173)) - session.id is deprecated. Instead, use dfs.metrics.session-id 2025-07-05 01:36:43,462 INFO [main] jvm.JvmMetrics (JvmMetrics.java:init(76)) - Initializing JVM Metrics with processName=JobTracker, sessionId= 2025-07-05 01:36:44,783 INFO [main] db.DBInputFormat (DBInputFormat.java:setTxIsolation(192)) - Using read commited transaction isolation 2025-07-05 01:36:44,854 INFO [main] mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(198)) - number of splits:1 2025-07-05 01:36:45,148 INFO [main] mapreduce.JobSubmitter (JobSubmitter.java:printTokens(287)) - Submitting tokens for job: job_local745453984_0001 2025-07-05 01:36:46,724 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605463/jackson-mapper-asl-1.9.13.jar <- /home/ljm1/jackson-mapper-asl-1.9.13.jar 2025-07-05 01:36:46,755 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/jackson-mapper-asl-1.9.13.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605463/jackson- 2025-07-05 01:36:46,793 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605464/kite-data-core-1.0.0.jar <- /home/ljm1/kite-data-core-1.0.0.jar 2025-07-05 01:36:46,798 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/kite-data-core-1.0.0.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605464/kite-data-cor 2025-07-05 01:36:46,798 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605465/xz-1.0.jar <- /home/ljm1/xz-1.0.jar 2025-07-05 01:36:46,801 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/xz-1.0.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605465/xz-1.0.jar 2025-07-05 01:36:46,801 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605466/kite-hadoop-compatibility-1.0.0.jar <- /home/ljm1/kite-hadoop-compatibility-1.0.0.j 2025-07-05 01:36:46,805 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/kite-hadoop-compatibility-1.0.0.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605466/ki 2025-07-05 01:36:46,805 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605467/parquet-avro-1.4.1.jar <- /home/ljm1/parquet-avro-1.4.1.jar 2025-07-05 01:36:46,810 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/parquet-avro-1.4.1.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605467/parquet-avro-1. 2025-07-05 01:36:46,811 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605468/sqoop-1.4.6.jar <- /home/ljm1/sqoop-1.4.6.jar 2025-07-05 01:36:46,818 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/sqoop-1.4.6.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605468/sqoop-1.4.6.jar 2025-07-05 01:36:46,818 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605469/kite-data-hive-1.0.0.jar <- /home/ljm1/kite-data-hive-1.0.0.jar 2025-07-05 01:36:46,829 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/kite-data-hive-1.0.0.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605469/kite-data-hiv 2025-07-05 01:36:46,829 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605470/parquet-generator-1.4.1.jar <- /home/ljm1/parquet-generator-1.4.1.jar 2025-07-05 01:36:46,833 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/parquet-generator-1.4.1.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605470/parquet-ge 2025-07-05 01:36:46,833 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605471/hsqldb-1.8.0.10.jar <- /home/ljm1/hsqldb-1.8.0.10.jar 2025-07-05 01:36:46,840 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/hsqldb-1.8.0.10.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605471/hsqldb-1.8.0.10.ja 2025-07-05 01:36:46,840 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605472/jackson-annotations-2.3.0.jar <- /home/ljm1/jackson-annotations-2.3.0.jar 2025-07-05 01:36:46,843 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/jackson-annotations-2.3.0.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605472/jackson- 2025-07-05 01:36:46,844 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605473/avro-1.7.5.jar <- /home/ljm1/avro-1.7.5.jar 2025-07-05 01:36:46,849 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/avro-1.7.5.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605473/avro-1.7.5.jar 2025-07-05 01:36:46,849 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605474/ant-contrib-1.0b3.jar <- /home/ljm1/ant-contrib-1.0b3.jar 2025-07-05 01:36:46,852 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/ant-contrib-1.0b3.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605474/ant-contrib-1.0b 2025-07-05 01:36:46,853 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605475/commons-compress-1.4.1.jar <- /home/ljm1/commons-compress-1.4.1.jar 2025-07-05 01:36:46,856 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/commons-compress-1.4.1.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605475/commons-com 2025-07-05 01:36:46,856 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605476/mysql-connector-java-5.1.47.jar <- /home/ljm1/mysql-connector-java-5.1.47.jar 2025-07-05 01:36:46,864 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/mysql-connector-java-5.1.47.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605476/mysql- 2025-07-05 01:36:46,865 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605477/slf4j-api-1.6.1.jar <- /home/ljm1/slf4j-api-1.6.1.jar 2025-07-05 01:36:46,869 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/slf4j-api-1.6.1.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605477/slf4j-api-1.6.1.ja 2025-07-05 01:36:46,869 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605478/opencsv-2.3.jar <- /home/ljm1/opencsv-2.3.jar 2025-07-05 01:36:46,873 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/opencsv-2.3.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605478/opencsv-2.3.jar 2025-07-05 01:36:46,874 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605479/parquet-column-1.4.1.jar <- /home/ljm1/parquet-column-1.4.1.jar 2025-07-05 01:36:46,878 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/parquet-column-1.4.1.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605479/parquet-colum 2025-07-05 01:36:46,879 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605480/jackson-core-asl-1.9.13.jar <- /home/ljm1/jackson-core-asl-1.9.13.jar 2025-07-05 01:36:46,882 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/jackson-core-asl-1.9.13.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605480/jackson-co 2025-07-05 01:36:46,882 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605481/snappy-java-1.0.5.jar <- /home/ljm1/snappy-java-1.0.5.jar 2025-07-05 01:36:46,888 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/snappy-java-1.0.5.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605481/snappy-java-1.0. 2025-07-05 01:36:46,889 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605482/commons-codec-1.4.jar <- /home/ljm1/commons-codec-1.4.jar 2025-07-05 01:36:46,893 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/commons-codec-1.4.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605482/commons-codec-1. 2025-07-05 01:36:46,893 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605483/commons-logging-1.1.1.jar <- /home/ljm1/commons-logging-1.1.1.jar 2025-07-05 01:36:46,897 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/commons-logging-1.1.1.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605483/commons-logg 2025-07-05 01:36:46,898 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605484/commons-io-1.4.jar <- /home/ljm1/commons-io-1.4.jar 2025-07-05 01:36:46,902 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/commons-io-1.4.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605484/commons-io-1.4.jar 2025-07-05 01:36:46,903 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605485/commons-jexl-2.1.1.jar <- /home/ljm1/commons-jexl-2.1.1.jar 2025-07-05 01:36:46,906 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/commons-jexl-2.1.1.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605485/commons-jexl-2. 2025-07-05 01:36:46,907 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605486/jackson-databind-2.3.1.jar <- /home/ljm1/jackson-databind-2.3.1.jar 2025-07-05 01:36:46,913 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/jackson-databind-2.3.1.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605486/jackson-dat 2025-07-05 01:36:46,913 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605487/parquet-jackson-1.4.1.jar <- /home/ljm1/parquet-jackson-1.4.1.jar 2025-07-05 01:36:46,916 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/parquet-jackson-1.4.1.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605487/parquet-jack 2025-07-05 01:36:46,916 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605488/paranamer-2.3.jar <- /home/ljm1/paranamer-2.3.jar 2025-07-05 01:36:46,919 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/paranamer-2.3.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605488/paranamer-2.3.jar 2025-07-05 01:36:46,920 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605489/parquet-common-1.4.1.jar <- /home/ljm1/parquet-common-1.4.1.jar 2025-07-05 01:36:46,924 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/parquet-common-1.4.1.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605489/parquet-commo 2025-07-05 01:36:46,925 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605490/avro-mapred-1.7.5-hadoop2.jar <- /home/ljm1/avro-mapred-1.7.5-hadoop2.jar 2025-07-05 01:36:46,929 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/avro-mapred-1.7.5-hadoop2.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605490/avro-map 2025-07-05 01:36:46,930 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605491/ant-eclipse-1.0-jvm1.2.jar <- /home/ljm1/ant-eclipse-1.0-jvm1.2.jar 2025-07-05 01:36:46,932 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/ant-eclipse-1.0-jvm1.2.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605491/ant-eclipse 2025-07-05 01:36:46,933 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605492/parquet-hadoop-1.4.1.jar <- /home/ljm1/parquet-hadoop-1.4.1.jar 2025-07-05 01:36:46,943 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/parquet-hadoop-1.4.1.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605492/parquet-hadoo 2025-07-05 01:36:46,943 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605493/mysql-connector-java-5.1.32.jar <- /home/ljm1/mysql-connector-java-5.1.32.jar 2025-07-05 01:36:46,947 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/mysql-connector-java-5.1.32.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605493/mysql- 2025-07-05 01:36:46,947 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605494/parquet-format-2.0.0.jar <- /home/ljm1/parquet-format-2.0.0.jar 2025-07-05 01:36:46,950 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/parquet-format-2.0.0.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605494/parquet-forma 2025-07-05 01:36:46,950 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605495/jackson-core-2.3.1.jar <- /home/ljm1/jackson-core-2.3.1.jar 2025-07-05 01:36:46,960 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/jackson-core-2.3.1.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605495/jackson-core-2. 2025-07-05 01:36:46,960 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605496/parquet-encoding-1.4.1.jar <- /home/ljm1/parquet-encoding-1.4.1.jar 2025-07-05 01:36:46,966 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/parquet-encoding-1.4.1.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605496/parquet-enc 2025-07-05 01:36:46,967 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:sym/mapred/local/1751704605497/kite-data-mapreduce-1.0.0.jar <- /home/ljm1/kite-data-mapreduce-1.0.0.jar 2025-07-05 01:36:46,973 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setvers/sqoop-1.4.6/lib/kite-data-mapreduce-1.0.0.jar as file:/tmp/hadoop-ljm1/mapred/local/1751704605497/kite-dat 2025-07-05 01:36:47,158 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605463/jackson-mapper-asl-1.9.13.jar 2025-07-05 01:36:47,158 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605464/kite-data-core-1.0.0.jar 2025-07-05 01:36:47,158 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605465/xz-1.0.jar 2025-07-05 01:36:47,158 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605466/kite-hadoop-compatibility-1.0.0.jar 2025-07-05 01:36:47,158 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605467/parquet-avro-1.4.1.jar 2025-07-05 01:36:47,158 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605468/sqoop-1.4.6.jar 2025-07-05 01:36:47,158 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605469/kite-data-hive-1.0.0.jar 2025-07-05 01:36:47,158 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605470/parquet-generator-1.4.1.jar 2025-07-05 01:36:47,159 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605471/hsqldb-1.8.0.10.jar 2025-07-05 01:36:47,159 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605472/jackson-annotations-2.3.0.jar 2025-07-05 01:36:47,159 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605473/avro-1.7.5.jar 2025-07-05 01:36:47,159 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605474/ant-contrib-1.0b3.jar 2025-07-05 01:36:47,159 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605475/commons-compress-1.4.1.jar 2025-07-05 01:36:47,159 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605476/mysql-connector-java-5.1.47.jar 2025-07-05 01:36:47,159 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605477/slf4j-api-1.6.1.jar 2025-07-05 01:36:47,159 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605478/opencsv-2.3.jar 2025-07-05 01:36:47,159 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605479/parquet-column-1.4.1.jar 2025-07-05 01:36:47,159 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605480/jackson-core-asl-1.9.13.jar 2025-07-05 01:36:47,160 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605481/snappy-java-1.0.5.jar 2025-07-05 01:36:47,160 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605482/commons-codec-1.4.jar 2025-07-05 01:36:47,160 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605483/commons-logging-1.1.1.jar 2025-07-05 01:36:47,160 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605484/commons-io-1.4.jar 2025-07-05 01:36:47,160 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605485/commons-jexl-2.1.1.jar 2025-07-05 01:36:47,160 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605486/jackson-databind-2.3.1.jar 2025-07-05 01:36:47,160 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605487/parquet-jackson-1.4.1.jar 2025-07-05 01:36:47,160 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605488/paranamer-2.3.jar 2025-07-05 01:36:47,160 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605489/parquet-common-1.4.1.jar 2025-07-05 01:36:47,160 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605490/avro-mapred-1.7.5-hadoop2.jar 2025-07-05 01:36:47,160 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605491/ant-eclipse-1.0-jvm1.2.jar 2025-07-05 01:36:47,160 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605492/parquet-hadoop-1.4.1.jar 2025-07-05 01:36:47,161 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605493/mysql-connector-java-5.1.32.jar 2025-07-05 01:36:47,188 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605494/parquet-format-2.0.0.jar 2025-07-05 01:36:47,188 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605495/jackson-core-2.3.1.jar 2025-07-05 01:36:47,188 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605496/parquet-encoding-1.4.1.jar 2025-07-05 01:36:47,188 INFO [main] mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:maked/local/1751704605497/kite-data-mapreduce-1.0.0.jar 2025-07-05 01:36:47,213 INFO [main] mapreduce.Job (Job.java:submit(1294)) - The url to track the job: http://l 2025-07-05 01:36:47,214 INFO [main] mapreduce.Job (Job.java:monitorAndPrintJob(1339)) - Running job: job_local 2025-07-05 01:36:47,220 INFO [Thread-44] mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471) 2025-07-05 01:36:47,468 INFO [Thread-44] output.FileOutputCommitter (FileOutputCommitter.java:<init>(108)) - F 2025-07-05 01:36:47,470 INFO [Thread-44] mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)uce.lib.output.FileOutputCommitter 2025-07-05 01:36:47,482 ERROR [Thread-44] output.FileOutputCommitter (FileOutputCommitter.java:setupJob(314)) -ucts/_temporary/0 2025-07-05 01:36:47,686 INFO [Thread-44] mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting f 2025-07-05 01:36:47,688 INFO [LocalJobRunner Map Task Executor #0] mapred.LocalJobRunner (LocalJobRunner.java:84_0001_m_000000_0 2025-07-05 01:36:47,791 INFO [LocalJobRunner Map Task Executor #0] output.FileOutputCommitter (FileOutputCommiAlgorithm version is 1 2025-07-05 01:36:47,828 INFO [LocalJobRunner Map Task Executor #0] mapred.Task (Task.java:initialize(612)) - 2025-07-05 01:36:47,919 INFO [LocalJobRunner Map Task Executor #0] db.DBInputFormat (DBInputFormat.java:setTxIn isolation 2025-07-05 01:36:47,931 INFO [LocalJobRunner Map Task Executor #0] mapred.MapTask (MapTask.java:runNewMapper(7 2025-07-05 01:36:47,959 INFO [Thread-44] mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task 2025-07-05 01:36:47,966 WARN [Thread-44] mapred.LocalJobRunner (LocalJobRunner.java:run(560)) - job_local74545 java.lang.Exception: java.io.IOException: Mkdirs failed to create file:/user/hadoop/products/_temporary/0/_tempxists=false, cwd=file:/home/ljm1) at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522) Caused by: java.io.IOException: Mkdirs failed to create file:/user/hadoop/products/_temporary/0/_temporary/attee, cwd=file:/home/ljm1) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:455) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:440) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:911) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:892) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:789) at org.apache.sqoop.mapreduce.RawKeyTextOutputFormat.getRecordWriter(RawKeyTextOutputFormat.java:98) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.<init>(MapTask.java:647) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:767) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2025-07-05 01:36:48,217 INFO [main] mapreduce.Job (Job.java:monitorAndPrintJob(1360)) - Job job_local745453984 2025-07-05 01:36:48,221 INFO [main] mapreduce.Job (Job.java:monitorAndPrintJob(1367)) - map 0% reduce 0% 2025-07-05 01:36:48,226 INFO [main] mapreduce.Job (Job.java:monitorAndPrintJob(1380)) - Job job_local745453984 2025-07-05 01:36:48,258 INFO [main] mapreduce.Job (Job.java:monitorAndPrintJob(1385)) - Counters: 0 2025-07-05 01:36:48,265 WARN [main] mapreduce.Counters (AbstractCounters.java:getGroup(234)) - Group FileSystemapreduce.FileSystemCounter instead 2025-07-05 01:36:48,272 INFO [main] mapreduce.ImportJobBase (ImportJobBase.java:runJob(184)) - Transferred 0 b 2025-07-05 01:36:48,277 WARN [main] mapreduce.Counters (AbstractCounters.java:getGroup(234)) - Group org.apach org.apache.hadoop.mapreduce.TaskCounter instead 2025-07-05 01:36:48,277 INFO [main] mapreduce.ImportJobBase (ImportJobBase.java:runJob(186)) - Retrieved 0 rec 2025-07-05 01:36:48,277 ERROR [main] tool.ImportTool (ImportTool.java:run(621)) - Error during import: Import j 这是怎么回事
07-06
C:\Users\lenovo\.jdks\ms-17.0.15\bin\java.exe "-javaagent:D:\新建文件夹\IntelliJ IDEA 2023.3.4\lib\idea_rt.jar=49561:D:\新建文件夹\IntelliJ IDEA 2023.3.4\bin" -Dfile.encoding=UTF-8 -classpath D:\highStudy\WebMagic\target\classes;D:\apache-maven-3.6.1\mav-repo\io\github\bonigarcia\webdrivermanager\5.6.0\webdrivermanager-5.6.0.jar;D:\apache-maven-3.6.1\mav-repo\org\slf4j\slf4j-api\2.0.9\slf4j-api-2.0.9.jar;D:\apache-maven-3.6.1\mav-repo\com\google\code\gson\gson\2.10.1\gson-2.10.1.jar;D:\apache-maven-3.6.1\mav-repo\com\github\docker-java\docker-java\3.3.3\docker-java-3.3.3.jar;D:\apache-maven-3.6.1\mav-repo\com\github\docker-java\docker-java-core\3.3.3\docker-java-core-3.3.3.jar;D:\apache-maven-3.6.1\mav-repo\com\github\docker-java\docker-java-api\3.3.3\docker-java-api-3.3.3.jar;D:\apache-maven-3.6.1\mav-repo\org\bouncycastle\bcpkix-jdk18on\1.75\bcpkix-jdk18on-1.75.jar;D:\apache-maven-3.6.1\mav-repo\org\bouncycastle\bcprov-jdk18on\1.75\bcprov-jdk18on-1.75.jar;D:\apache-maven-3.6.1\mav-repo\org\bouncycastle\bcutil-jdk18on\1.75\bcutil-jdk18on-1.75.jar;D:\apache-maven-3.6.1\mav-repo\org\slf4j\jcl-over-slf4j\2.0.9\jcl-over-slf4j-2.0.9.jar;D:\apache-maven-3.6.1\mav-repo\com\github\docker-java\docker-java-transport-httpclient5\3.3.3\docker-java-transport-httpclient5-3.3.3.jar;D:\apache-maven-3.6.1\mav-repo\com\github\docker-java\docker-java-transport\3.3.3\docker-java-transport-3.3.3.jar;D:\apache-maven-3.6.1\mav-repo\net\java\dev\jna\jna\5.12.1\jna-5.12.1.jar;D:\apache-maven-3.6.1\mav-repo\org\brotli\dec\0.1.2\dec-0.1.2.jar;D:\apache-maven-3.6.1\mav-repo\org\apache\commons\commons-lang3\3.13.0\commons-lang3-3.13.0.jar;D:\apache-maven-3.6.1\mav-repo\org\apache\commons\commons-compress\1.24.0\commons-compress-1.24.0.jar;D:\apache-maven-3.6.1\mav-repo\org\apache\httpcomponents\client5\httpclient5\5.2.1\httpclient5-5.2.1.jar;D:\apache-maven-3.6.1\mav-repo\org\apache\httpcomponents\core5\httpcore5\5.2.3\httpcore5-5.2.3.jar;D:\apache-maven-3.6.1\mav-repo\org\apache\httpcomponents\core5\httpcore5-h2\5.2.3\httpcore5-h2-5.2.3.jar;D:\apache-maven-3.6.1\mav-repo\org\springframework\boot\spring-boot-starter\3.2.0\spring-boot-starter-3.2.0.jar;D:\apache-maven-3.6.1\mav-repo\org\springframework\boot\spring-boot\3.2.0\spring-boot-3.2.0.jar;D:\apache-maven-3.6.1\mav-repo\org\springframework\spring-context\6.1.1\spring-context-6.1.1.jar;D:\apache-maven-3.6.1\mav-repo\org\springframework\boot\spring-boot-autoconfigure\3.2.0\spring-boot-autoconfigure-3.2.0.jar;D:\apache-maven-3.6.1\mav-repo\org\springframework\boot\spring-boot-starter-logging\3.2.0\spring-boot-starter-logging-3.2.0.jar;D:\apache-maven-3.6.1\mav-repo\ch\qos\logback\logback-classic\1.4.11\logback-classic-1.4.11.jar;D:\apache-maven-3.6.1\mav-repo\ch\qos\logback\logback-core\1.4.11\logback-core-1.4.11.jar;D:\apache-maven-3.6.1\mav-repo\org\apache\logging\log4j\log4j-to-slf4j\2.21.1\log4j-to-slf4j-2.21.1.jar;D:\apache-maven-3.6.1\mav-repo\org\apache\logging\log4j\log4j-api\2.21.1\log4j-api-2.21.1.jar;D:\apache-maven-3.6.1\mav-repo\org\slf4j\jul-to-slf4j\2.0.9\jul-to-slf4j-2.0.9.jar;D:\apache-maven-3.6.1\mav-repo\jakarta\annotation\jakarta.annotation-api\2.1.1\jakarta.annotation-api-2.1.1.jar;D:\apache-maven-3.6.1\mav-repo\org\springframework\spring-core\6.1.1\spring-core-6.1.1.jar;D:\apache-maven-3.6.1\mav-repo\org\springframework\spring-jcl\6.1.1\spring-jcl-6.1.1.jar;D:\apache-maven-3.6.1\mav-repo\org\yaml\snakeyaml\2.2\snakeyaml-2.2.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-java\4.15.0\selenium-java-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-api\4.15.0\selenium-api-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-chrome-driver\4.15.0\selenium-chrome-driver-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\com\google\auto\service\auto-service-annotations\1.1.1\auto-service-annotations-1.1.1.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-chromium-driver\4.15.0\selenium-chromium-driver-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-json\4.15.0\selenium-json-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-manager\4.15.0\selenium-manager-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-devtools-v117\4.15.0\selenium-devtools-v117-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-devtools-v118\4.15.0\selenium-devtools-v118-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-devtools-v119\4.15.0\selenium-devtools-v119-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-devtools-v85\4.15.0\selenium-devtools-v85-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-edge-driver\4.15.0\selenium-edge-driver-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-firefox-driver\4.15.0\selenium-firefox-driver-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-http\4.15.0\selenium-http-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\dev\failsafe\failsafe\3.3.2\failsafe-3.3.2.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-ie-driver\4.15.0\selenium-ie-driver-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-remote-driver\4.15.0\selenium-remote-driver-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\com\google\guava\guava\32.1.2-jre\guava-32.1.2-jre.jar;D:\apache-maven-3.6.1\mav-repo\com\google\guava\failureaccess\1.0.1\failureaccess-1.0.1.jar;D:\apache-maven-3.6.1\mav-repo\com\google\guava\listenablefuture\9999.0-empty-to-avoid-conflict-with-guava\listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar;D:\apache-maven-3.6.1\mav-repo\com\google\code\findbugs\jsr305\3.0.2\jsr305-3.0.2.jar;D:\apache-maven-3.6.1\mav-repo\org\checkerframework\checker-qual\3.33.0\checker-qual-3.33.0.jar;D:\apache-maven-3.6.1\mav-repo\com\google\errorprone\error_prone_annotations\2.18.0\error_prone_annotations-2.18.0.jar;D:\apache-maven-3.6.1\mav-repo\com\google\j2objc\j2objc-annotations\2.8\j2objc-annotations-2.8.jar;D:\apache-maven-3.6.1\mav-repo\io\opentelemetry\opentelemetry-api\1.31.0\opentelemetry-api-1.31.0.jar;D:\apache-maven-3.6.1\mav-repo\io\opentelemetry\opentelemetry-context\1.31.0\opentelemetry-context-1.31.0.jar;D:\apache-maven-3.6.1\mav-repo\io\opentelemetry\opentelemetry-exporter-logging\1.31.0\opentelemetry-exporter-logging-1.31.0.jar;D:\apache-maven-3.6.1\mav-repo\io\opentelemetry\opentelemetry-sdk-common\1.31.0\opentelemetry-sdk-common-1.31.0.jar;D:\apache-maven-3.6.1\mav-repo\io\opentelemetry\opentelemetry-sdk-extension-autoconfigure-spi\1.31.0\opentelemetry-sdk-extension-autoconfigure-spi-1.31.0.jar;D:\apache-maven-3.6.1\mav-repo\io\opentelemetry\opentelemetry-sdk-extension-autoconfigure\1.31.0\opentelemetry-sdk-extension-autoconfigure-1.31.0.jar;D:\apache-maven-3.6.1\mav-repo\io\opentelemetry\opentelemetry-api-events\1.31.0-alpha\opentelemetry-api-events-1.31.0-alpha.jar;D:\apache-maven-3.6.1\mav-repo\io\opentelemetry\opentelemetry-sdk-trace\1.31.0\opentelemetry-sdk-trace-1.31.0.jar;D:\apache-maven-3.6.1\mav-repo\io\opentelemetry\opentelemetry-sdk\1.31.0\opentelemetry-sdk-1.31.0.jar;D:\apache-maven-3.6.1\mav-repo\io\opentelemetry\opentelemetry-sdk-metrics\1.31.0\opentelemetry-sdk-metrics-1.31.0.jar;D:\apache-maven-3.6.1\mav-repo\io\opentelemetry\opentelemetry-extension-incubator\1.31.0-alpha\opentelemetry-extension-incubator-1.31.0-alpha.jar;D:\apache-maven-3.6.1\mav-repo\io\opentelemetry\opentelemetry-sdk-logs\1.31.0\opentelemetry-sdk-logs-1.31.0.jar;D:\apache-maven-3.6.1\mav-repo\io\opentelemetry\opentelemetry-semconv\1.28.0-alpha\opentelemetry-semconv-1.28.0-alpha.jar;D:\apache-maven-3.6.1\mav-repo\net\bytebuddy\byte-buddy\1.14.10\byte-buddy-1.14.10.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-os\4.15.0\selenium-os-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\org\apache\commons\commons-exec\1.3\commons-exec-1.3.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-safari-driver\4.15.0\selenium-safari-driver-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\org\seleniumhq\selenium\selenium-support\4.15.0\selenium-support-4.15.0.jar;D:\apache-maven-3.6.1\mav-repo\org\springframework\boot\spring-boot-starter-web\3.2.0\spring-boot-starter-web-3.2.0.jar;D:\apache-maven-3.6.1\mav-repo\org\springframework\boot\spring-boot-starter-json\3.2.0\spring-boot-starter-json-3.2.0.jar;D:\apache-maven-3.6.1\mav-repo\com\fasterxml\jackson\core\jackson-databind\2.15.3\jackson-databind-2.15.3.jar;D:\apache-maven-3.6.1\mav-repo\com\fasterxml\jackson\core\jackson-annotations\2.15.3\jackson-annotations-2.15.3.jar;D:\apache-maven-3.6.1\mav-repo\com\fasterxml\jackson\core\jackson-core\2.15.3\jackson-core-2.15.3.jar;D:\apache-maven-3.6.1\mav-repo\com\fasterxml\jackson\datatype\jackson-datatype-jdk8\2.15.3\jackson-datatype-jdk8-2.15.3.jar;D:\apache-maven-3.6.1\mav-repo\com\fasterxml\jackson\datatype\jackson-datatype-jsr310\2.15.3\jackson-datatype-jsr310-2.15.3.jar;D:\apache-maven-3.6.1\mav-repo\com\fasterxml\jackson\module\jackson-module-parameter-names\2.15.3\jackson-module-parameter-names-2.15.3.jar;D:\apache-maven-3.6.1\mav-repo\org\springframework\boot\spring-boot-starter-tomcat\3.2.0\spring-boot-starter-tomcat-3.2.0.jar;D:\apache-maven-3.6.1\mav-repo\org\apache\tomcat\embed\tomcat-embed-core\10.1.16\tomcat-embed-core-10.1.16.jar;D:\apache-maven-3.6.1\mav-repo\org\apache\tomcat\embed\tomcat-embed-el\10.1.16\tomcat-embed-el-10.1.16.jar;D:\apache-maven-3.6.1\mav-repo\org\apache\tomcat\embed\tomcat-embed-websocket\10.1.16\tomcat-embed-websocket-10.1.16.jar;D:\apache-maven-3.6.1\mav-repo\org\springframework\spring-web\6.1.1\spring-web-6.1.1.jar;D:\apache-maven-3.6.1\mav-repo\org\springframework\spring-beans\6.1.1\spring-beans-6.1.1.jar;D:\apache-maven-3.6.1\mav-repo\io\micrometer\micrometer-observation\1.12.0\micrometer-observation-1.12.0.jar;D:\apache-maven-3.6.1\mav-repo\io\micrometer\micrometer-commons\1.12.0\micrometer-commons-1.12.0.jar;D:\apache-maven-3.6.1\mav-repo\org\springframework\spring-webmvc\6.1.1\spring-webmvc-6.1.1.jar;D:\apache-maven-3.6.1\mav-repo\org\springframework\spring-aop\6.1.1\spring-aop-6.1.1.jar;D:\apache-maven-3.6.1\mav-repo\org\springframework\spring-expression\6.1.1\spring-expression-6.1.1.jar;D:\apache-maven-3.6.1\mav-repo\org\projectlombok\lombok\1.18.30\lombok-1.18.30.jar;D:\apache-maven-3.6.1\mav-repo\com\mysql\mysql-connector-j\8.1.0\mysql-connector-j-8.1.0.jar;D:\apache-maven-3.6.1\mav-repo\us\codecraft\webmagic-core\0.8.0\webmagic-core-0.8.0.jar;D:\apache-maven-3.6.1\mav-repo\us\codecraft\xsoup\0.3.6\xsoup-0.3.6.jar;D:\apache-maven-3.6.1\mav-repo\org\apache\commons\commons-collections4\4.4\commons-collections4-4.4.jar;D:\apache-maven-3.6.1\mav-repo\commons-io\commons-io\2.11.0\commons-io-2.11.0.jar;D:\apache-maven-3.6.1\mav-repo\com\jayway\jsonpath\json-path\2.8.0\json-path-2.8.0.jar;D:\apache-maven-3.6.1\mav-repo\com\alibaba\fastjson\2.0.19.graal\fastjson-2.0.19.graal.jar;D:\apache-maven-3.6.1\mav-repo\com\alibaba\fastjson2\fastjson2-extension\2.0.19.graal\fastjson2-extension-2.0.19.graal.jar;D:\apache-maven-3.6.1\mav-repo\com\alibaba\fastjson2\fastjson2\2.0.19.graal\fastjson2-2.0.19.graal.jar;D:\apache-maven-3.6.1\mav-repo\us\codecraft\webmagic-selenium\0.8.0\webmagic-selenium-0.8.0.jar;D:\apache-maven-3.6.1\mav-repo\com\github\detro\phantomjsdriver\1.2.0\phantomjsdriver-1.2.0.jar;D:\apache-maven-3.6.1\mav-repo\us\codecraft\webmagic-extension\0.8.0\webmagic-extension-0.8.0.jar;D:\apache-maven-3.6.1\mav-repo\redis\clients\jedis\5.0.2\jedis-5.0.2.jar;D:\apache-maven-3.6.1\mav-repo\org\apache\commons\commons-pool2\2.12.0\commons-pool2-2.12.0.jar;D:\apache-maven-3.6.1\mav-repo\org\json\json\20231013\json-20231013.jar;D:\apache-maven-3.6.1\mav-repo\org\jsoup\jsoup\1.17.2\jsoup-1.17.2.jar;D:\apache-maven-3.6.1\mav-repo\net\minidev\json-smart\2.5.0\json-smart-2.5.0.jar;D:\apache-maven-3.6.1\mav-repo\net\minidev\accessors-smart\2.5.0\accessors-smart-2.5.0.jar;D:\apache-maven-3.6.1\mav-repo\org\ow2\asm\asm\9.3\asm-9.3.jar;D:\apache-maven-3.6.1\mav-repo\org\assertj\assertj-core\3.24.2\assertj-core-3.24.2.jar;D:\apache-maven-3.6.1\mav-repo\org\springframework\boot\spring-boot-configuration-processor\3.2.0\spring-boot-configuration-processor-3.2.0.jar;D:\apache-maven-3.6.1\mav-repo\org\apache\httpcomponents\httpclient\4.5.14\httpclient-4.5.14.jar;D:\apache-maven-3.6.1\mav-repo\org\apache\httpcomponents\httpcore\4.4.16\httpcore-4.4.16.jar;D:\apache-maven-3.6.1\mav-repo\commons-logging\commons-logging\1.2\commons-logging-1.2.jar;D:\apache-maven-3.6.1\mav-repo\commons-codec\commons-codec\1.16.0\commons-codec-1.16.0.jar com.gzc.WebMagicStudy 15:08:09.076 [main] INFO us.codecraft.webmagic.Spider -- Spider www.weather.com.cn started! java.io.FileNotFoundException: \data\webmagic\webmagic-selenium\config.ini (系统找不到指定的路径。) at java.base/java.io.FileInputStream.open0(Native Method) at java.base/java.io.FileInputStream.open(FileInputStream.java:216) at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157) at java.base/java.io.FileInputStream.<init>(FileInputStream.java:111) at java.base/java.io.FileReader.<init>(FileReader.java:60) at us.codecraft.webmagic.downloader.selenium.WebDriverPool.configure(WebDriverPool.java:72) at us.codecraft.webmagic.downloader.selenium.WebDriverPool.get(WebDriverPool.java:197) at us.codecraft.webmagic.downloader.selenium.SeleniumDownloader.download(SeleniumDownloader.java:79) at us.codecraft.webmagic.Spider.processRequest(Spider.java:445) at us.codecraft.webmagic.Spider.access$000(Spider.java:65) at us.codecraft.webmagic.Spider$1.run(Spider.java:349) at us.codecraft.webmagic.thread.CountableThreadPool$1.run(CountableThreadPool.java:74) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840)
12-02
基于TROPOMI高光谱遥感仪器获取的大气成分观测资料,本研究聚焦于大气污染物一氧化氮(NO₂)的空间分布与浓度定量反演问题。NO₂作为影响空气质量的关键指标,其精确监测对环境保护与大气科学研究具有显著价值。当前,利用卫星遥感数据结合先进算法实现NO₂浓度的高精度反演已成为该领域的重要研究方向。 本研究构建了一套以深度学习为核心的技术框架,整合了来自TROPOMI仪器的光谱辐射信息、观测几何参数以及辅助气象数据,形成多维度特征数据集。该数据集充分融合了不同来源的观测信息,为深入解析大气中NO₂的时空变化规律提供了数据基础,有助于提升反演模型的准确性与环境预测的可靠性。 在模型架构方面,项目设计了一种多分支神经网络,用于分别处理光谱特征与气象特征等多模态数据。各分支通过独立学习提取代表性特征,并在深层网络中进行特征融合,从而综合利用不同数据的互补信息,显著提高了NO₂浓度反演的整体精度。这种多源信息融合策略有效增强了模型对复杂大气环境的表征能力。 研究过程涵盖了系统的数据处理流程。前期预处理包括辐射定标、噪声抑制及数据标准化等步骤,以保障输入特征的质量与一致性;后期处理则涉及模型输出的物理量转换与结果验证,确保反演结果符合实际大气浓度范围,提升数据的实用价值。 此外,本研究进一步对不同功能区域(如城市建成区、工业带、郊区及自然背景区)的NO₂浓度分布进行了对比分析,揭示了人类活动与污染物空间格局的关联性。相关结论可为区域环境规划、污染管控政策的制定提供科学依据,助力大气环境治理与公共健康保护。 综上所述,本研究通过融合TROPOMI高光谱数据与多模态特征深度学习技术,发展了一套高效、准确的大气NO₂浓度遥感反演方法,不仅提升了卫星大气监测的技术水平,也为环境管理与决策支持提供了重要的技术工具。 资源来源于网络分享,仅用于学习交流使用,请勿用于商业,如有侵权请联系我删除!
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值