OkHttp再分析<一>

为啥要有再次分析OkHttp呢,因为作为android 网络底层框架,它的使用范围和稳定性都是非常高的。那么我们需要对他进行充分的熟悉。
我们还是从他自身的同步和异步请求方法来分析吧,因为这个是落脚方法。位于NewCall里面。

  @Override public Response execute() throws IOException {
  	//首先锁住自身,这个synchronized是一个非重入锁。如果同一线程或者别的线程想再次调用对象的execute 方法,executed 是个布尔变量,线程的可见性可以保障。如果发现正在执行就会抛出异常。
    synchronized (this) {
      if (executed) throw new IllegalStateException("Already Executed");
      executed = true;
    }
    captureCallStackTrace();
    timeout.enter();
    eventListener.callStart(this);
    try {
      //还是利用Dispatch 来执行这个RealCall.那么下面我们着重分析这个。
      client.dispatcher().executed(this);
      Response result = getResponseWithInterceptorChain();
      if (result == null) throw new IOException("Canceled");
      return result;
    } catch (IOException e) {
      e = timeoutExit(e);
      eventListener.callFailed(this, e);
      throw e;
    } finally {
      client.dispatcher().finished(this);
    }
  }

Dispatcher的executed 实际也只是把他加入ArrayDeque 这个双向队列,跟ArrayList一样,不过也是线程不安全的,所以加上了锁。

synchronized void executed(RealCall call) {
    runningSyncCalls.add(call);
  }

那么执行方法应该就是getResponseWithInterceptorChain()了,我们来分析下。

  Response getResponseWithInterceptorChain() throws IOException {
    // Build a full stack of interceptors.
    List<Interceptor> interceptors = new ArrayList<>();
    //自己传入进去的拦截器。
    interceptors.addAll(client.interceptors());
    interceptors.add(retryAndFollowUpInterceptor);
    interceptors.add(new BridgeInterceptor(client.cookieJar()));
    interceptors.add(new CacheInterceptor(client.internalCache()));
    interceptors.add(new ConnectInterceptor(client));
    if (!forWebSocket) {
      interceptors.addAll(client.networkInterceptors());
    }
    //这个拦截器是最终执行请求的。一会看下怎么通过责任链的模式,请求到这里。
    interceptors.add(new CallServerInterceptor(forWebSocket));

    Interceptor.Chain chain = new RealInterceptorChain(interceptors, null, null, null, 0,
        originalRequest, this, eventListener, client.connectTimeoutMillis(),
        client.readTimeoutMillis(), client.writeTimeoutMillis());

    return chain.proceed(originalRequest);
  }

落脚方法跑到了RealInterceptorChain的proceed,可以猜测这个是把所有拦截器统一起来的方法,看看他的名字Real。开始分析proceed。

public Response proceed(Request request, StreamAllocation streamAllocation, HttpCodec httpCodec,
      RealConnection connection) throws IOException {
      //index代表正在执行的拦截器的索引。
    if (index >= interceptors.size()) throw new AssertionError();
    
    calls++;

    // If we already have a stream, confirm that the incoming request will use it.
    //httpCode不为空代表正在请求。
    if (this.httpCodec != null && !this.connection.supportsUrl(request.url())) {
      throw new IllegalStateException("network interceptor " + interceptors.get(index - 1)
          + " must retain the same host and port");
    }

    // If we already have a stream, confirm that this is the only call to chain.proceed().
    if (this.httpCodec != null && calls > 1) {
      throw new IllegalStateException("network interceptor " + interceptors.get(index - 1)
          + " must call proceed() exactly once");
    }

    // Call the next interceptor in the chain.
    RealInterceptorChain next = new RealInterceptorChain(interceptors, streamAllocation, httpCodec,
        connection, index + 1, request, call, eventListener, connectTimeout, readTimeout,
        writeTimeout);
    //这个地方实际上就是代表把当前的后一个传入进去,然后对当前的拦截器调用intercept方法。因为每个拦截器都会有一个方法就是    (RealInterceptorChain)chain.proceed 这样保证拦截器都能够执行到intercept方法同时也能下一个执行到,这个算法还是值得一看的。
    Interceptor interceptor = interceptors.get(index);
    Response response = interceptor.intercept(next);

    // Confirm that the next interceptor made its required call to chain.proceed().
    if (httpCodec != null && index + 1 < interceptors.size() && next.calls != 1) {
      throw new IllegalStateException("network interceptor " + interceptor
          + " must call proceed() exactly once");
    }

    // Confirm that the intercepted response isn't null.
    if (response == null) {
      throw new NullPointerException("interceptor " + interceptor + " returned null");
    }

    if (response.body() == null) {
      throw new IllegalStateException(
          "interceptor " + interceptor + " returned a response with no body");
    }

    return response;
  }

那么我们还是一个一个从OkHttp的拦截来说吧,因为他为实现http请求做的一些列工作。retryAndFollowUpInterceptor属于重复请求的,暂时不管。
BridgeInterceptor

  @Override public Response intercept(Chain chain) throws IOException {
    Request userRequest = chain.request();
    Request.Builder requestBuilder = userRequest.newBuilder();

    RequestBody body = userRequest.body();
    //如果是post请求,RequestBody 有数据,如果是get就没有。有个类似的概念叫做ReponseBody 我们返回的数据可以在这个里面去获取。
    if (body != null) {
      MediaType contentType = body.contentType();
      //添加 Content-Type 到header。
      if (contentType != null) {
        requestBuilder.header("Content-Type", contentType.toString());
      }
       
      long contentLength = body.contentLength();
      if (contentLength != -1) {
        requestBuilder.header("Content-Length", Long.toString(contentLength));
        requestBuilder.removeHeader("Transfer-Encoding");
      } else {
        requestBuilder.header("Transfer-Encoding", "chunked");
        requestBuilder.removeHeader("Content-Length");
      }
    }

    if (userRequest.header("Host") == null) {
      requestBuilder.header("Host", hostHeader(userRequest.url(), false));
    }

    if (userRequest.header("Connection") == null) {
      requestBuilder.header("Connection", "Keep-Alive");
    }

    // If we add an "Accept-Encoding: gzip" header field we're responsible for also decompressing
    // the transfer stream.
    boolean transparentGzip = false;
    if (userRequest.header("Accept-Encoding") == null && userRequest.header("Range") == null) {
      transparentGzip = true;
      requestBuilder.header("Accept-Encoding", "gzip");
    }
    //添加Cookie.
    List<Cookie> cookies = cookieJar.loadForRequest(userRequest.url());
    if (!cookies.isEmpty()) {
      requestBuilder.header("Cookie", cookieHeader(cookies));
    }
    
    if (userRequest.header("User-Agent") == null) {
      requestBuilder.header("User-Agent", Version.userAgent());
    }
    //传递出去给到下面拦截器。
    Response networkResponse = chain.proceed(requestBuilder.build());
    //实际这一步是保存Cookie信息需要更进去看。
    HttpHeaders.receiveHeaders(cookieJar, userRequest.url(), networkResponse.headers());

    Response.Builder responseBuilder = networkResponse.newBuilder()
        .request(userRequest);

    if (transparentGzip
        && "gzip".equalsIgnoreCase(networkResponse.header("Content-Encoding"))
        && HttpHeaders.hasBody(networkResponse)) {
      GzipSource responseBody = new GzipSource(networkResponse.body().source());
      Headers strippedHeaders = networkResponse.headers().newBuilder()
          .removeAll("Content-Encoding")
          .removeAll("Content-Length")
          .build();
      responseBuilder.headers(strippedHeaders);
      String contentType = networkResponse.header("Content-Type");
      responseBuilder.body(new RealResponseBody(contentType, -1L, Okio.buffer(responseBody)));
    }

    return responseBuilder.build();
  }

可以看到主要是做了一些Cook和header的设置工作。
接下来看下面一个。CacheInterceptor 主要是跟缓存相关。

  @Override public Response intercept(Chain chain) throws IOException {
  //首先分析此request的缓存是否存在,就是同样的请求的Response是否存在。
    Response cacheCandidate = cache != null
        ? cache.get(chain.request())
        : null;
    
    long now = System.currentTimeMillis();
     //获取到CacheStrategy。 
    CacheStrategy strategy = new CacheStrategy.Factory(now, chain.request(), cacheCandidate).get();
    Request networkRequest = strategy.networkRequest;
    Response cacheResponse = strategy.cacheResponse;

    if (cache != null) {
      cache.trackResponse(strategy);
    }

    if (cacheCandidate != null && cacheResponse == null) {
      closeQuietly(cacheCandidate.body()); // The cache candidate wasn't applicable. Close it.
    }

    // If we're forbidden from using the network and the cache is insufficient, fail.
    //networkRequest 代表网络请求,如果这个不存在那么直接返回了,不会继续往下走了。
    if (networkRequest == null && cacheResponse == null) {
      return new Response.Builder()
          .request(chain.request())
          .protocol(Protocol.HTTP_1_1)
          .code(504)
          .message("Unsatisfiable Request (only-if-cached)")
          .body(Util.EMPTY_RESPONSE)
          .sentRequestAtMillis(-1L)
          .receivedResponseAtMillis(System.currentTimeMillis())
          .build();
    }

    // If we don't need the network, we're done.
    //如果网络请求不存在,但是缓存存在直接利用缓存的Reponse.
    if (networkRequest == null) {
      return cacheResponse.newBuilder()
          .cacheResponse(stripBody(cacheResponse))
          .build();
    }
     //不利用缓存开始请求了。 
    Response networkResponse = null;
    try {
      networkResponse = chain.proceed(networkRequest);
    } finally {
      // If we're crashing on I/O or otherwise, don't leak the cache body.
      if (networkResponse == null && cacheCandidate != null) {
        closeQuietly(cacheCandidate.body());
      }
    }

    // If we have a cache response too, then we're doing a conditional get.
    //更新缓存。
    if (cacheResponse != null) {
      if (networkResponse.code() == HTTP_NOT_MODIFIED) {
        Response response = cacheResponse.newBuilder()
            .headers(combine(cacheResponse.headers(), networkResponse.headers()))
            .sentRequestAtMillis(networkResponse.sentRequestAtMillis())
            .receivedResponseAtMillis(networkResponse.receivedResponseAtMillis())
            .cacheResponse(stripBody(cacheResponse))
            .networkResponse(stripBody(networkResponse))
            .build();
        networkResponse.body().close();

        // Update the cache after combining headers but before stripping the
        // Content-Encoding header (as performed by initContentStream()).
        cache.trackConditionalCacheHit();
        cache.update(cacheResponse, response);
        return response;
      } else {
        closeQuietly(cacheResponse.body());
      }
    }

    Response response = networkResponse.newBuilder()
        .cacheResponse(stripBody(cacheResponse))
        .networkResponse(stripBody(networkResponse))
        .build();

    if (cache != null) {
      if (HttpHeaders.hasBody(response) && CacheStrategy.isCacheable(response, networkRequest)) {
        // Offer this request to the cache.
        CacheRequest cacheRequest = cache.put(response);
        return cacheWritingResponse(cacheRequest, response);
      }
     //验证下缓存是否合法。否则移除。
      if (HttpMethod.invalidatesCache(networkRequest.method())) {
        try {
          cache.remove(networkRequest);
        } catch (IOException ignored) {
          // The cache cannot be written.
        }
      }
    }

    return response;
  }

接下来ConnectInterceptor。这个也是一个对连接产生直接影响的类还是非常短的。

  @Override public Response intercept(Chain chain) throws IOException {
    RealInterceptorChain realChain = (RealInterceptorChain) chain;
    Request request = realChain.request();
    //这个StreamAllocation 是在RetryAndFollowUpInterceptor创建的
    //    StreamAllocation streamAllocation = new StreamAllocation(client.connectionPool(),
    //    createAddress(request.url()), call, eventListener, callStackTrace);
  
   StreamAllocation streamAllocation = realChain.streamAllocation();
    // We need the network to satisfy this request. Possibly for validating a conditional GET.
    boolean doExtensiveHealthChecks = !request.method().equals("GET");
    //获取到HttpCodec对象
    HttpCodec httpCodec = streamAllocation.newStream(client, chain, doExtensiveHealthChecks);
    //会复用RealConnection.
    RealConnection connection = streamAllocation.connection();

    return realChain.proceed(request, streamAllocation, httpCodec, connection);
  }

接下来来到最后一个CallServerInterceptor。内容比较多,打算再开一篇文章来分析。

package org.example; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.ObjectMapper; import okhttp3.*; import org.apache.spark.sql.Dataset; import org.apache.spark.sql.Row; import org.apache.spark.sql.RowFactory; import org.apache.spark.sql.SparkSession; import org.apache.spark.sql.catalyst.analysis.NoSuchTableException; import org.apache.spark.sql.types.DataTypes; import org.apache.spark.sql.types.StructType; import java.io.IOException; import java.time.LocalDate; import java.time.format.DateTimeFormatter; import java.util.*; /** * Title: TakenToiceberg * Author: hyx * Package: org.example * Date: 2025/10/13 10:05 * Description: 从API拉取数据并写入Iceberg */ public class TakenToiceberg { private static final String LOGIN_URL = "http://172.16.3.40:8002/api/Login/Login"; private static final String READ_DATA_URL = "http://172.16.3.40:8002/api/ReadNowData_TEMP/Get"; private static final String ALARM_DATA_URL = "http://172.16.3.40:8002/api/DeviceAlarmNow/GetAlarmInfoByAlarmStatus"; private static final String USERNAME = "admin"; private static final String PASSWORD = "Rlxs@123456."; private static final String DEVICE_ADDR = "502412100404"; // 可配置为列表 private static final OkHttpClient httpClient = new OkHttpClient(); private static final ObjectMapper objectMapper = new ObjectMapper(); public static void main(String[] args) { SparkSession spark = SparkSession.builder() .appName("DeviceDataToIceberg") .master("local[*]") .config("spark.sql.catalog.iceberg_catalog", "org.apache.iceberg.spark.SparkCatalog") .config("spark.sql.catalog.iceberg_catalog.type", "hive") .config("spark.sql.catalog.iceberg_catalog.warehouse", "hdfs://10.62.167.57:8020/user/iceberg/warehouse/reli_collect.db") .config("spark.hadoop.dfs.nameservices", "bjrl") .config("spark.hadoop.dfs.ha.namenodes.bjrl", "nn1,nn2") .config("spark.hadoop.dfs.namenode.rpc-address.bjrl.nn1", "10.62.167.51:8020") .config("spark.hadoop.dfs.namenode.rpc-address.bjrl.nn2", "10.62.167.52:8020") .config("spark.hadoop.dfs.client.failover.proxy.provider.bjrl", "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider") .getOrCreate(); try { // Step 1: 登录获取Token System.out.println("===== Step 1: 开始登录获取Token ====="); String token = loginAndGetToken(); System.out.println("Token获取成功,前20位:" + (token != null ? token.substring(0, 20) + "..." : "(Token为null)")); // Step 2: 拉取设备数据 System.out.println("===== Step 2: 开始拉取设备数据 ====="); JsonNode deviceData = fetchDeviceData(token, DEVICE_ADDR); if (deviceData != null && deviceData.has("data") && deviceData.get("data").has("data")) { List<JsonNode> dataList = new ArrayList<>(); deviceData.get("data").get("data").forEach(dataList::add); System.out.println("成功拉取 " + dataList.size() + " 条设备数据"); writeDeviceDataToIceberg(spark, dataList); } else { System.out.println("警告:API返回的设备数据为空或格式不匹配,跳过写入"); } // Step 3: 拉取告警数据(今日) System.out.println("===== Step 3: 开始拉取今日告警数据 ====="); String today = LocalDate.now().format(DateTimeFormatter.ofPattern("yyyy-MM-dd")); JsonNode alarmData = fetchAlarmData(token, DEVICE_ADDR, today, today); if (alarmData != null && alarmData.has("data") && alarmData.get("data").has("data")) { List<JsonNode> alarmList = new ArrayList<>(); alarmData.get("data").get("data").forEach(alarmList::add); System.out.println("成功拉取 " + alarmList.size() + " 条告警数据"); writeAlarmDataToIceberg(spark, alarmList); } else { System.out.println("警告:API返回的告警数据为空或格式不匹配,跳过写入"); } } catch (Exception e) { System.err.println("执行过程中发生异常:"); e.printStackTrace(); } finally { System.out.println("===== 正在停止Spark会话 ====="); spark.stop(); System.out.println("Spark会话已停止"); } } /** * 登录并获取Token */ private static String loginAndGetToken() throws IOException { MediaType JSON = MediaType.get("application/json; charset=utf-8"); String jsonBody = String.format( "{\"Guid\":\"null\",\"username\":\"%s\",\"password\":\"%s\",\"VerificationCode\":\"1234\"}", USERNAME, PASSWORD ); RequestBody body = RequestBody.create(JSON, jsonBody); Request request = new Request.Builder() .url(LOGIN_URL) .post(body) .addHeader("Content-Type", "application/json") .build(); try (Response response = httpClient.newCall(request).execute()) { if (!response.isSuccessful()) { throw new IOException("登录请求失败,响应码:" + response.code() + ",响应消息:" + response.message()); } JsonNode root = objectMapper.readTree(response.body().string()); // 防御性判断:确保success、data、token字段存在且合法 if (root.has("success") && root.get("success").asBoolean() && root.has("data") && root.get("data").has("success") && root.get("data").get("success").asBoolean() && root.get("data").has("token")) { return root.get("data").get("token").asText(); } else { throw new RuntimeException("登录响应格式非法,响应内容:" + root); } } } /** * 拉取设备实时数据 */ private static JsonNode fetchDeviceData(String token, String deviceAddr) throws IOException { HttpUrl url = HttpUrl.parse(READ_DATA_URL) .newBuilder() .addQueryParameter("status", "0") .addQueryParameter("deptId", "0") .addQueryParameter("keyType", "deviceAddr") .addQueryParameter("key", deviceAddr) .build(); Request request = new Request.Builder() .url(url) .addHeader("Authorization", "Bearer " + token) .get() .build(); try (Response response = httpClient.newCall(request).execute()) { if (!response.isSuccessful()) { throw new IOException("拉取设备数据失败,响应码:" + response.code() + ",响应消息:" + response.message()); } return objectMapper.readTree(response.body().string()); } } /** * 拉取告警数据 */ private static JsonNode fetchAlarmData(String token, String deviceAddr, String begin, String end) throws IOException { HttpUrl url = HttpUrl.parse(ALARM_DATA_URL) .newBuilder() .addQueryParameter("pageNo", "1") .addQueryParameter("pageSize", "100") .addQueryParameter("deptId", "0") .addQueryParameter("keyType", "deviceAddr") .addQueryParameter("key", deviceAddr) .addQueryParameter("orderBy", "AlarmEndTime") .addQueryParameter("orderByType", "desc") .addQueryParameter("beginTime", begin) .addQueryParameter("endTime", end) .addQueryParameter("alarmStatus", "0") .build(); Request request = new Request.Builder() .url(url) .addHeader("Authorization", "Bearer " + token) .get() .build(); try (Response response = httpClient.newCall(request).execute()) { if (!response.isSuccessful()) { throw new IOException("拉取告警数据失败,响应码:" + response.code() + ",响应消息:" + response.message()); } return objectMapper.readTree(response.body().string()); } } /** * 将设备数据写入Iceberg */ private static void writeDeviceDataToIceberg(SparkSession spark, List<JsonNode> dataList) throws NoSuchTableException { List<Row> rows = new ArrayList<>(); for (JsonNode node : dataList) { // 每个字段都做空值防御性处理 String deviceAddr = node.has("DeviceAddr") && !node.get("DeviceAddr").isNull() ? node.get("DeviceAddr").asText() : null; String monitorName = node.has("MonitorName") && !node.get("MonitorName").isNull() ? node.get("MonitorName").asText() : null; String deviceTime = node.has("DeviceTime") && !node.get("DeviceTime").isNull() ? node.get("DeviceTime").asText() : null; Double temperature = node.has("Temperature") && !node.get("Temperature").isNull() ? node.get("Temperature").asDouble() : null; Double temperature2 = node.has("Temperature2") && !node.get("Temperature2").isNull() ? node.get("Temperature2").asDouble() : null; Integer liquidPosition = node.has("LiquidPosition") && !node.get("LiquidPosition").isNull() ? node.get("LiquidPosition").asInt() : null; Double voltage = node.has("Voltage") && !node.get("Voltage").isNull() ? node.get("Voltage").asDouble() : null; Integer csq = node.has("CSQ") && !node.get("CSQ").isNull() ? node.get("CSQ").asInt() : null; String imei = node.has("IMEI") && !node.get("IMEI").isNull() ? node.get("IMEI").asText() : null; String version = node.has("Version") && !node.get("Version").isNull() ? node.get("Version").asText() : null; String electricQuantity = node.has("ElectricQuantity") && !node.get("ElectricQuantity").isNull() ? node.get("ElectricQuantity").asText() : null; Boolean isOnline = node.has("IsOnline") && !node.get("IsOnline").isNull() ? node.get("IsOnline").asBoolean() : null; Boolean isAlarm = node.has("IsAlarm") && !node.get("IsAlarm").isNull() ? node.get("IsAlarm").asBoolean() : null; rows.add(RowFactory.create( deviceAddr, monitorName, deviceTime, temperature, temperature2, liquidPosition, voltage, csq, imei, version, electricQuantity, isOnline, isAlarm )); } StructType schema = new StructType() .add("device_addr", DataTypes.StringType) .add("monitor_name", DataTypes.StringType) .add("device_time", DataTypes.StringType) .add("temperature", DataTypes.DoubleType) .add("temperature2", DataTypes.DoubleType) .add("liquid_position", DataTypes.IntegerType) .add("voltage", DataTypes.DoubleType) .add("csq", DataTypes.IntegerType) .add("imei", DataTypes.StringType) .add("version", DataTypes.StringType) .add("electric_quantity", DataTypes.StringType) .add("is_online", DataTypes.BooleanType) .add("is_alarm", DataTypes.BooleanType); Dataset<Row> df = spark.createDataFrame(rows, schema); System.out.println("开始写入设备数据到Iceberg表:iceberg_catalog.reli_collect.ods1_d_cus_device_data"); df.writeTo("iceberg_catalog.reli_collect.ods1_d_cus_device_data").append(); System.out.println("设备数据写入完成,共 " + rows.size() + " 条"); } /** * 将告警数据写入Iceberg */ private static void writeAlarmDataToIceberg(SparkSession spark, List<JsonNode> alarmList) throws NoSuchTableException { List<Row> rows = new ArrayList<>(); for (JsonNode node : alarmList) { // 每个字段都做空值防御性处理 String deviceAddr = node.has("DeviceAddr") && !node.get("DeviceAddr").isNull() ? node.get("DeviceAddr").asText() : null; String deviceName = node.has("DeviceName") && !node.get("DeviceName").isNull() ? node.get("DeviceName").asText() : null; String monitorName = node.has("MonitorName") && !node.get("MonitorName").isNull() ? node.get("MonitorName").asText() : null; String alarmStartTime = node.has("AlarmStartTime") && !node.get("AlarmStartTime").isNull() ? node.get("AlarmStartTime").asText() : null; String alarmEndTime = node.has("AlarmEndTime") && !node.get("AlarmEndTime").isNull() ? node.get("AlarmEndTime").asText() : null; String alarmMsg = node.has("DeviceAlarmStr") && !node.get("DeviceAlarmStr").isNull() ? node.get("DeviceAlarmStr").asText() : null; String alarmMsg2 = node.has("DeviceAlarmStr2") && !node.get("DeviceAlarmStr2").isNull() ? node.get("DeviceAlarmStr2").asText() : null; Integer deviceState = node.has("DeviceState") && !node.get("DeviceState").isNull() ? node.get("DeviceState").asInt() : null; String readTime = node.has("ReadTime") && !node.get("ReadTime").isNull() ? node.get("ReadTime").asText() : null; rows.add(RowFactory.create( deviceAddr, deviceName, monitorName, alarmStartTime, alarmEndTime, alarmMsg, alarmMsg2, deviceState, readTime )); } StructType schema = new StructType() .add("device_addr", DataTypes.StringType) .add("device_name", DataTypes.StringType) .add("monitor_name", DataTypes.StringType) .add("alarm_start_time", DataTypes.StringType) .add("alarm_end_time", DataTypes.StringType) .add("alarm_msg", DataTypes.StringType) .add("alarm_msg2", DataTypes.StringType) .add("device_state", DataTypes.IntegerType) .add("read_time", DataTypes.StringType); Dataset<Row> df = spark.createDataFrame(rows, schema); System.out.println("开始写入告警数据到Iceberg表:iceberg_catalog.reli_collect.ods1_d_cus_device_alarms"); df.writeTo("iceberg_catalog.reli_collect.ods1_d_cus_device_alarms").append(); System.out.println("告警数据写入完成,共 " + rows.size() + " 条"); } } <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <!-- 1. 项目基础信息(替换为你的GroupId/ArtifactId) --> <groupId>org.example</groupId> <artifactId>thermal-data-to-iceberg</artifactId> <version>1.0-SNAPSHOT</version> <properties> <maven.compiler.source>1.8</maven.compiler.source> <maven.compiler.target>1.8</maven.compiler.target> <spark.version>3.5.0</spark.version> <iceberg.version>1.4.0</iceberg.version> <jackson.version>2.15.2</jackson.version> <okhttp.version>4.11.0</okhttp.version> </properties> <dependencies> <!-- Spark: provided --> <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-sql_2.12</artifactId> <version>${spark.version}</version> <scope>provided</scope> </dependency> <!-- Iceberg: 必须打包!不要 provided --> <dependency> <groupId>org.apache.iceberg</groupId> <artifactId>iceberg-spark-runtime-3.5_2.12</artifactId> <version>${iceberg.version}</version> <!-- 注意:没有 scope --> </dependency> <!-- 其他工具依赖 --> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId> <version>${jackson.version}</version> </dependency> <dependency> <groupId>com.squareup.okhttp3</groupId> <artifactId>okhttp</artifactId> <version>${okhttp.version}</version> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-shade-plugin</artifactId> <version>3.4.1</version> <executions> <execution> <phase>package</phase> <goals> <goal>shade</goal> </goals> <configuration> <createDependencyReducedPom>false</createDependencyReducedPom> <filters> <filter> <artifact>*:*</artifact> <excludes> <exclude>META-INF/*.SF</exclude> <exclude>META-INF/*.DSA</exclude> <exclude>META-INF/*.RSA</exclude> </excludes> </filter> </filters> <transformers> <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"> <mainClass>org.example.TokenToiceberg</mainClass> <!-- 确保正确! --> </transformer> </transformers> </configuration> </execution> </executions> </plugin> <!-- compiler plugin ... --> </plugins> </build> </project>分析代码和依赖是否正确
10-16
2025-06-10 23:59:24.729 30065-4066 DataLoader com.example.kucun2 E Failed to load data javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found. at com.android.org.conscrypt.SSLUtils.toSSLHandshakeException(SSLUtils.java:356) at com.android.org.conscrypt.ConscryptEngine.convertException(ConscryptEngine.java:1127) at com.android.org.conscrypt.ConscryptEngine.readPlaintextData(ConscryptEngine.java:1082) at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:869) at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:740) at com.android.org.conscrypt.ConscryptEngine.unwrap(ConscryptEngine.java:705) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.processDataFromSocket(ConscryptEngineSocket.java:896) at com.android.org.conscrypt.ConscryptEngineSocket$SSLInputStream.-$$Nest$mprocessDataFromSocket(Unknown Source:0) at com.android.org.conscrypt.ConscryptEngineSocket.doHandshake(ConscryptEngineSocket.java:236) at com.android.org.conscrypt.ConscryptEngineSocket.startHandshake(ConscryptEngineSocket.java:218) at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.kt:379) at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.kt:337) at okhttp3.internal.connection.RealConnection.connect(RealConnection.kt:209) at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.kt:226) at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.kt:106) at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.kt:74) at okhttp3.internal.connection.RealCall.initExchange$okhttp(RealCall.kt:255) <string name="url">https://192.168.31.177:3000</string> <string name="url_all">/app/all</string> <string name="url_bancis">/app/bancai/all</string> <string name="url_caizhis">/app/caizhi/all</string> <string name="url_mupis">/app/mupi/all</string> <string name="url_dingdans">/app/dingdan/all</string> <string name="url_chanpins">/app/chanpin/all</string> <string name="url_zujians">/app/zujian/all</string> <string name="url_chanpin_zujians">/app/chanpin_zujian/all</string> <string name="url_dingdan_zujians">/app/dingdan_zujian/all</string> <string name="url_dingdan_chanpins">/app/dingdan_chanpin/all</string> <string name="url_jinhuos">/app/jinhuo/all</string> <string name="url_add_bancai">/app/bancai/add</string> <string name="url_add_dingdan">/app/dingdan/add</string> <string name="url_add_chanpin">/app/chanpin/add</string> <string name="url_add_zujian">/app/zujian/add</string> <string name="url_add_caizhi">/app/caizhi/add</string> <string name="url_add_mupi">/app/mupi/add</string> <string name="url_add_dingdan_chanpin">/app/dingdan_chanpi/add</string> <string name="url_add_dingdan_zujian">/app/dingdan_zujian/add</string> <string name="url_add_chanpin_zujian">/app/chanpin_zujian/add</string> <string name="url_add_jinhuo">/app/jinhuo/add</string> package com.example.kucun2.entity.data; import android.content.Context; import android.util.Log; import com.example.kucun2.entity.*; import com.example.kucun2.function.MyAppFnction; import com.google.gson.Gson; import com.google.gson.reflect.TypeToken; import java.io.IOException; import java.lang.reflect.Type; import java.util.*; import okhttp3.Call; import okhttp3.Callback; import okhttp3.OkHttpClient; import okhttp3.Request; import okhttp3.Response; public class Data { // 数据集合声明(保持原有属性名不变) public static SynchronizedList<Bancai> bancais = new SynchronizedList<>(Bancai.class); public static SynchronizedList<Caizhi> caizhis = new SynchronizedList<>(Caizhi.class); public static SynchronizedList<Mupi> mupis = new SynchronizedList<>(Mupi.class); public static SynchronizedList<Chanpin> chanpins = new SynchronizedList<>(Chanpin.class); public static SynchronizedList<Chanpin_Zujian> chanpinZujians = new SynchronizedList<>(Chanpin_Zujian.class); public static SynchronizedList<Dingdan> dingdans = new SynchronizedList<>(Dingdan.class); public static SynchronizedList<Dingdan_Chanpin> dingdanChanpins = new SynchronizedList<>(Dingdan_Chanpin.class); public static SynchronizedList<Dingdan_chanpin_zujian> dingdanBancais = new SynchronizedList<>(Dingdan_chanpin_zujian.class); public static SynchronizedList<Kucun> kucuns = new SynchronizedList<>(Kucun.class); public static SynchronizedList<Zujian> zujians = new SynchronizedList<>(Zujian.class); public static SynchronizedList<User> users = new SynchronizedList<>(Bancai.class); public static SynchronizedList<Jinhuo> jinhuoList = new SynchronizedList<>(Bancai.class); private static final Gson gson = new Gson(); private static final OkHttpClient client = new OkHttpClient(); private static final String TAG = "DataLoader"; // 加载所有数据的方法 public static void loadAllData(Context context, LoadDataCallback callback) { String url = MyAppFnction.getStringResource("string", "url") + MyAppFnction.getStringResource("string", "url_all"); Log.d("url", "loadAllData: "+url); Request request = new Request.Builder() .url(url) .build(); client.newCall(request).enqueue(new Callback() { @Override public void onFailure(Call call, IOException e) { Log.e(TAG, "Failed to load data", e); if (callback != null) callback.onFailure(); } @Override public void onResponse(Call call, Response response) throws IOException { if (!response.isSuccessful()) { Log.e(TAG, "Unexpected response code: " + response.code()); if (callback != null) callback.onFailure(); return; } String responseData = response.body().string(); parseAndAssignData(responseData, context, callback); } }); } // 解析并赋值数据 private static void parseAndAssignData(String jsonData, Context context, LoadDataCallback callback) { // 创建包含所有数据类型的TypeToken Type informationType = new TypeToken<Information<AllDataResponse>>(){}.getType(); Information<AllDataResponse> information = gson.fromJson(jsonData, informationType); if (information == null || information.getData() == null || information.getStatus() != 200) { Log.e(TAG, "Invalid response data"); if (callback != null) callback.onFailure(); return; } AllDataResponse allData = information.getData(); // 赋值到对应的列表(使用安全方法保持已有引用) updateList(bancais, allData.bancais); updateList(caizhis, allData.caizhis); updateList(mupis, allData.mupis); updateList(chanpins, allData.chanpins); updateList(chanpinZujians, allData.chanpin_zujians); updateList(dingdans, allData.dingdans); updateList(dingdanChanpins, allData.dingdan_chanpins); updateList(dingdanBancais, allData.dingdan_bancais); updateList(kucuns, allData.kucuns); updateList(zujians, allData.zujians); updateList(users, allData.users); updateList(jinhuoList, allData.jinhuos); resolveReferences(); if (callback != null) callback.onSuccess(); } // 更新列表内容但保持对象引用不变 private static <T> void updateList(List<T> existingList, List<T> newList) { if (newList == null) return; existingList.clear(); existingList.addAll(newList); } // 解析对象间的引用关系(使用ID关联) private static void resolveReferences() { // 创建ID到对象的映射 Map<Integer, Bancai> bancaiMap = createIdMap(bancais); Map<Integer, Caizhi> caizhiMap = createIdMap(caizhis); Map<Integer, Mupi> mupiMap = createIdMap(mupis); Map<Integer, Chanpin> chanpinMap = createIdMap(chanpins); Map<Integer, Zujian> zujianMap = createIdMap(zujians); Map<Integer, Dingdan> dingdanMap = createIdMap(dingdans); Map<Integer, Kucun> kucunMap = createIdMap(kucuns); // 解析Bancai引用 for (Bancai bancai : bancais) { if (bancai.getCaizhi() != null) { bancai.setCaizhi(caizhiMap.get(bancai.getCaizhi().getId())); } if (bancai.getMupi1() != null) { bancai.setMupi1(mupiMap.get(bancai.getMupi1().getId())); } if (bancai.getMupi2() != null) { bancai.setMupi2(mupiMap.get(bancai.getMupi2().getId())); } } // 解析Kucun引用 for (Kucun kucun : kucuns) { if (kucun.getBancai() != null) { kucun.setBancai(bancaiMap.get(kucun.getBancai().getId())); } } // 解析订单相关的嵌套对象 for (Dingdan_Chanpin dc : dingdanChanpins) { if (dc.getDingdan() != null) { dc.setDingdan(dingdanMap.get(dc.getDingdan().getId())); } if (dc.getChanpin() != null) { dc.setChanpin(chanpinMap.get(dc.getChanpin().getId())); } } // 解析产品相关的嵌套对象 for (Chanpin_Zujian cz : chanpinZujians) { if (cz.getChanpin() != null) { cz.setChanpin(chanpinMap.get(cz.getChanpin().getId())); } if (cz.getZujian() != null) { cz.setZujian(zujianMap.get(cz.getZujian().getId())); } if (cz.getBancai() != null) { cz.setBancai(bancaiMap.get(cz.getBancai().getId())); } } // 解析库存关联 for (Kucun k : kucuns) { if (k.getBancai() != null) { k.setBancai(bancaiMap.get(k.getBancai().getId())); } } } // 创建ID到对象的映射 private static <T extends EntityClassGrassrootsid> Map<Integer, T> createIdMap(List<T> list) { Map<Integer, T> map = new HashMap<>(); for (T item : list) { map.put(item.getId(), item); } return map; } // 回调接口 public interface LoadDataCallback { void onSuccess(); void onFailure(); } // 内部类用于解析JSON响应 public static class AllDataResponse { public List<Bancai> bancais; public List<Caizhi> caizhis; public List<Mupi> mupis; public List<Chanpin> chanpins; public List<Chanpin_Zujian> chanpin_zujians; public List<Dingdan> dingdans; public List<Dingdan_Chanpin> dingdan_chanpins; public List<Dingdan_chanpin_zujian> dingdan_bancais; public List<Kucun> kucuns; public List<Zujian> zujians; public List<User> users; public List<Jinhuo> jinhuos; } }
06-12
<dependencies> <!-- SpringBoot Web容器 --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.apache.pdfbox</groupId> <artifactId>pdfbox</artifactId> <version>2.0.29</version> </dependency> <dependency> <groupId>org.json</groupId> <artifactId>json</artifactId> <version>20231013</version> </dependency> <!-- HanLP --> <dependency> <groupId>com.hankcs</groupId> <artifactId>hanlp</artifactId> <version>portable-1.8.4</version> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <scope>provided</scope> </dependency> <dependency> <groupId>com.luxsan</groupId> <artifactId>lingxi-ai-common-core</artifactId> </dependency> <!-- Tess4J核心库 --> <dependency> <groupId>net.sourceforge.tess4j</groupId> <artifactId>tess4j</artifactId> <version>5.8.0</version> </dependency> <dependency> <groupId>org.openpnp</groupId> <artifactId>opencv</artifactId> <version>4.5.5-1</version> </dependency> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId> </dependency> <dependency> <groupId>com.fasterxml.jackson.dataformat</groupId> <artifactId>jackson-dataformat-xml</artifactId> <version>2.18.3</version> </dependency> <dependency> <groupId>org.json</groupId> <artifactId>json</artifactId> <version>20231013</version> </dependency> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpclient</artifactId> <version>4.5.13</version> </dependency> <!-- <dependency>--> <!-- <groupId>javax.media</groupId>--> <!-- <artifactId>jai_core</artifactId>--> <!-- <version>1.1.3</version>--> <!-- </dependency>--> <dependency> <groupId>jakarta.xml.bind</groupId> <artifactId>jakarta.xml.bind-api</artifactId> <version>4.0.0</version> </dependency> <dependency> <groupId>me.zhyd.oauth</groupId> <artifactId>JustAuth</artifactId> </dependency> <dependency> <groupId>com.squareup.okhttp3</groupId> <artifactId>okhttp</artifactId> <version>4.9.3</version> </dependency> <dependency> <groupId>dev.langchain4j</groupId> <artifactId>langchain4j</artifactId> <version>0.29.1</version> </dependency> <dependency> <groupId>dev.langchain4j</groupId> <artifactId>langchain4j-dashscope</artifactId> <version>0.29.1</version> </dependency> <dependency> <groupId>dev.langchain4j</groupId> <artifactId>langchain4j-core</artifactId> <version>0.29.1</version> </dependency> <!-- <dependency>--> <!-- <groupId>org.scala-lang</groupId>--> <!-- <artifactId>scala-library</artifactId>--> <!-- <version>2.13.9</version>--> <!-- <scope>compile</scope>--> <!-- </dependency>--> </dependencies>哪个冲突了
07-31
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值