The First Day_Beginning

There are three basic steps to SQL tuning:

1.Figure out which execution plan (path to reach the data your SQL statement demands) you are getting.

2.Change SQL or the database to get a chosen execution plan.

3.Figure out which execution plan is best.

In another word:

1.Which execution plan is best, and how can you find it without trial and error?

2.How does the current execution plan differ from the ideal execution plan, if it does?

3.If the difference between the actual and ideal execution plans is enough to matter, how can you change some combination of the SQL and the database to get close enough to the ideal execution plan for the performance that you need?

4.Does the new execution plan deliver the SQL performance you needed and expected?
[color=red]
benefit:
the only really significant part of the problem, deciding which execution plan is best, is virtually independent of our choice of relational database. The best execution plan is still the best execution plan.
[/color]
#!/usr/bin/env python3 # -*- coding: utf-8 -*- import os from osgeo import gdal, osr def process_mersi_data(input_file, geo_file, output_file, band_index=8, resolution=1000, lat_0=0, lon_0=105, lat_1=25, lat_2=47, resample_method=gdal.GRIORA_Bilinear): """ 处理FY3D-MERSI数据并进行地理校正与投影转换 参数: input_file: 待处理的HDF文件路径(包含EV_250_Aggr.1KM_Emissive) geo_file: 地理坐标HDF文件路径(包含Geolocation/Latitude和Longitude) output_file: 输出的GeoTIFF文件路径 band_index: 要处理的波段索引(默认:8,对应EV_250_Aggr.1KM_Emissive) resolution: 输出数据分辨率(米),默认1000米 lat_0, lon_0: Albers投影中心点经纬度(默认:0, 105) lat_1, lat_2: Albers投影标准纬线(默认:25, 47) resample_method: 重采样算法(默认:双线性插值) """ # 打开输入数据集 dataset = gdal.Open(input_file) # 获取子数据集列表并检查波段索引有效性 sub_datasets = dataset.GetSubDatasets() if band_index >= len(sub_datasets): raise ValueError(f"波段索引{band_index}超出范围,可用索引0-{len(sub_datasets) - 1}") # 创建临时VRT文件路径 base_name = os.path.splitext(os.path.basename(input_file))[0] vrt_path = os.path.join(os.path.dirname(input_file), f"{base_name}_band{band_index}.vrt") # 转换目标波段为VRT格式(EV_250_Aggr.1KM_Emissive) sub_dataset_path = sub_datasets[band_index][0] vrt_dataset = gdal.Translate(vrt_path, sub_dataset_path, format="VRT") if vrt_dataset is None: raise RuntimeError(f"创建VRT文件失败: {vrt_path}") del vrt_dataset # 关闭数据集以确保文件写入完成 # 构建Albers投影参数 srs = osr.SpatialReference() proj4_params = ( f"+proj=aea +lat_0={lat_0} +lon_0={lon_0} " f"+lat_1={lat_1} +lat_2={lat_2} +x_0=0 +y_0=0 " "+datum=WGS84 +units=m +no_defs" ) srs.ImportFromProj4(proj4_params) # 生成地理定位元数据块(关键修改:指定正确的经纬度路径) geo_metadata = [ '<Metadata domain="GEOLOCATION">', ' <MDI key="LINE_OFFSET">1</MDI>', ' <MDI key="LINE_STEP">1</MDI>', ' <MDI key="PIXEL_OFFSET">1</MDI>', ' <MDI key="PIXEL_STEP">1</MDI>', ' <MDI key="SRS">GEOGCS["WGS84",DATUM["WGS_1984",SPHEROID["WGS84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],TOWGS84[0,0,0,0,0,0,0],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9108"]],AUTHORITY["EPSG","4326"]]</MDI>', ' <MDI key="X_BAND">1</MDI>', # 重要!根据地理坐标文件的子数据集名称修改路径 f' <MDI key="X_DATASET">HDF5:"{geo_file}"://Geolocation/Longitude</MDI>', ' <MDI key="Y_BAND">1</MDI>', f' <MDI key="Y_DATASET">HDF5:"{geo_file}"://Geolocation/Latitude</MDI>', '</Metadata>' ] # 读取VRT文件内容并注入元数据 with open(vrt_path, 'r') as f: vrt_lines = f.readlines() vrt_lines.insert(1, '\n'.join(geo_metadata) + '\n') # 在第二行前插入元数据 with open(vrt_path, 'w') as f: f.writelines(vrt_lines) # 配置投影转换参数 warp_options = gdal.WarpOptions( format='GTiff', dstSRS=srs, xRes=resolution, yRes=resolution, resampleAlg=resample_method, creationOptions=[ 'COMPRESS=LZW', # LZW压缩减少文件大小 'TILED=YES', # 分块存储提高访问效率 'BIGTIFF=IF_NEEDED' # 支持大文件 ], multithread=True # 启用多线程处理 ) # 执行投影转换 try: warped_dataset = gdal.Warp(output_file, vrt_path, options=warp_options) if warped_dataset is None: raise RuntimeError("地理校正失败") del warped_dataset # 关闭数据集确保写入完成 finally: # 清理临时VRT文件 if os.path.exists(vrt_path): os.remove(vrt_path) print(f"处理完成: {output_file}") return True if __name__ == "__main__": # 设置文件路径(根据实际情况修改) input_file = r"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF" geo_file = r"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_GEO1K_MS.HDF" output_file = r"D:\FYData\Data\Q202506121354284980\0905.tif" # 设置处理参数 band_index = 8 # 对应EV_250_Aggr.1KM_Emissive数据集 resolution = 1000 # 输出分辨率(米) lon_0 = 105 # 投影中心经度 lat_1 = 25 # 标准纬线1 lat_2 = 47 # 标准纬线2 try: process_mersi_data( input_file=input_file, geo_file=geo_file, output_file=output_file, band_index=band_index, resolution=resolution, lon_0=lon_0, lat_1=lat_1, lat_2=lat_2 ) except Exception as e: print(f"处理过程中发生错误: {str(e)}")在这个基础上增加去除bowties的功能,下面是两个HDF文件的元数据和子数据集:Files: D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF Size is 512, 512 Metadata: AdditionalAnnotation=MERSI Geolocation use GPS data. AscendingNodeLongitude=33179878 BB_Count_Contaminated_Scans=0 Calibration_BB_DN_average_band_name=1-25 Calibration_BB_DN_average_FillValue=65535 Calibration_BB_DN_average_Intercept=0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Calibration_BB_DN_average_long_name=BlackBody Scanning DN Average Calibration_BB_DN_average_Slope=1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Calibration_BB_DN_average_units=none Calibration_BB_DN_average_valid_range=0 4095 Calibration_EV_start_time_band_name=none Calibration_EV_start_time_FillValue=65535 Calibration_EV_start_time_Intercept=0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Calibration_EV_start_time_long_name=Earth View Start Time Calibration_EV_start_time_Slope=1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Calibration_EV_start_time_units=second Calibration_EV_start_time_valid_range=0 876000 Calibration_Frame_Count_band_name=none Calibration_Frame_Count_FillValue=-2147483647 Calibration_Frame_Count_Intercept=0 Calibration_Frame_Count_long_name=Frame Count Calibration_Frame_Count_Slope=1 Calibration_Frame_Count_units=none Calibration_Frame_Count_valid_range=0 1 Calibration_IR_Cal_Coeff_band_name=20-25 Calibration_IR_Cal_Coeff_Description=Calibration Coefficients for thermal Emissive Bands and each scan. Radiance=k0+k1*DN+k2*DN^2+k3*DN^3. , the first dimension represents the 6 IR Bands , the second dimension contains the value of k0, k1 k2 and k3, and the third dimension response the scans. Calibration_IR_Cal_Coeff_FillValue=65535 Calibration_IR_Cal_Coeff_Intercept=0 Calibration_IR_Cal_Coeff_long_name=Emissive Bands calibration Coefficient Calibration_IR_Cal_Coeff_Slope=1 Calibration_IR_Cal_Coeff_units=none Calibration_IR_Cal_Coeff_valid_range=32767 -32767 Calibration_Kmirror_Side_band_name=none Calibration_Kmirror_Side_FillValue=255 Calibration_Kmirror_Side_Intercept=0 Calibration_Kmirror_Side_long_name=Kmirror Side(A or B side) Flag Calibration_Kmirror_Side_Slope=1 Calibration_Kmirror_Side_units=none Calibration_Kmirror_Side_valid_range=0 1 Calibration_Parameter_Revision_Date=2017-11-25 Calibration_SV_DN_average_band_name=1-25 Calibration_SV_DN_average_FillValue=65535 Calibration_SV_DN_average_Intercept=0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Calibration_SV_DN_average_long_name=Space View DN Average Calibration_SV_DN_average_Slope=1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Calibration_SV_DN_average_units=none Calibration_SV_DN_average_valid_range=0 4095 Calibration_VIS_Cal_Coeff_band_name=1-19 Calibration_VIS_Cal_Coeff_Description=Calibration Coefficients for Reflective Solar Bands. Reflectance=k0+k1*DN+k2*DN^2, the first dimension represents the 19 VIS Bands , and the second dimension contains the value of k0, k1 and k2. Calibration_VIS_Cal_Coeff_FillValue=65535 Calibration_VIS_Cal_Coeff_Intercept=0 Calibration_VIS_Cal_Coeff_long_name=Reflective Solar Bands Calibration Coefficients Calibration_VIS_Cal_Coeff_Slope=1 Calibration_VIS_Cal_Coeff_units=none Calibration_VIS_Cal_Coeff_valid_range=32767 -32767 Calibration_VOC_DN_average_band_name=1-25 Calibration_VOC_DN_average_FillValue=65535 Calibration_VOC_DN_average_Intercept=0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Calibration_VOC_DN_average_long_name=Mean Value of Visible Onboard Calibrator DN Calibration_VOC_DN_average_Slope=1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Calibration_VOC_DN_average_units=none Calibration_VOC_DN_average_valid_range=0 4095 Count_CaliErr_Scans=0 Count_GeolErr_Scans=4 Dataset_Name=Global MERSI Data Data_Creating_Date=2025-06-05 Data_Creating_Time=11:35:31.000 Data_EV_1KM_Emissive_band_name=20,21,22,23 Data_EV_1KM_Emissive_Description=Earth View Radiance for 1km Emissive Bands Note: =65535, data missing; =65534, detector is saturated; = 65533 detector is dead Data_EV_1KM_Emissive_FillValue=65535 Data_EV_1KM_Emissive_Intercept=0 0 0 0 Data_EV_1KM_Emissive_long_name=1km Emissive Bands Earth View Science Data Data_EV_1KM_Emissive_Slope=0.00019999999 0.00019999999 0.0099999998 0.0099999998 Data_EV_1KM_Emissive_units=mW/ (m2 cm-1 sr) Data_EV_1KM_Emissive_valid_range=0 65530 Data_EV_1KM_RefSB_band_name=5-19 Data_EV_1KM_RefSB_Description=Earth View Raw Data for1km Reflective solar Bands;Note: =65535, data missing; =65534, detector is saturated; = 65533 detector is dead. Data_EV_1KM_RefSB_FillValue=65535 Data_EV_1KM_RefSB_Intercept=0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Data_EV_1KM_RefSB_long_name=1km Reflective Bands Earth View Science Data Data_EV_1KM_RefSB_Slope=1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 Data_EV_1KM_RefSB_units=none Data_EV_1KM_RefSB_valid_range=0 4095 Data_EV_250_Aggr.1KM_Emissive_band_name=24,25 Data_EV_250_Aggr.1KM_Emissive_Description=250m Emissive Bands Earth View Radiance Data Aggregated to 1 km.Note: =65535, data missing; =65534, detector is saturated; = 65533 detector is dead. Data_EV_250_Aggr.1KM_Emissive_FillValue=65535 Data_EV_250_Aggr.1KM_Emissive_Intercept=0 0 Data_EV_250_Aggr.1KM_Emissive_long_name=250m Emissive Bands Earth View Science Data Aggregated to 1 km Data_EV_250_Aggr.1KM_Emissive_Slope=0.0099999998 0.0099999998 Data_EV_250_Aggr.1KM_Emissive_units=mW/ (m2 cm-1 sr) Data_EV_250_Aggr.1KM_Emissive_valid_range=0 25000 Data_EV_250_Aggr.1KM_RefSB_band_name=1,2,3,4 Data_EV_250_Aggr.1KM_RefSB_Description=250m Reflective Bands Earth View Raw Data Aggregated to 1 km;Note: =65535, data missing; =65534, detector is saturated; = 65533 detector is dead. Data_EV_250_Aggr.1KM_RefSB_FillValue=65535 Data_EV_250_Aggr.1KM_RefSB_Intercept=0 0 0 0 Data_EV_250_Aggr.1KM_RefSB_long_name=250m Reflective Bands Earth View Science Data Aggregated to 1 km Data_EV_250_Aggr.1KM_RefSB_Slope=1 1 1 1 Data_EV_250_Aggr.1KM_RefSB_units=none Data_EV_250_Aggr.1KM_RefSB_valid_range=0 4095 Data_Integrity=0 Day_Or_Night_Flag=D DN_Normalized_LUT_UpdateDate= DN_Normalized_LUT_version=V 1.0.1 EarthSun_Distance_Ratio=1.01457519141879 Eccentricity=267488 Effect_Center_WaveLength=0.47 0.55000001 0.64999998 0.86500001 1.38 1.64 2.1300001 0.412 0.44299999 0.49000001 0.55500001 0.67000002 0.70899999 0.74599999 0.86500001 0.90499997 0.93599999 0.94 1.03 3.796 4.046 7.2329998 8.5600004 10.714 11.948 EpochTime=250605 File_Alias_Name=MERSI_1KM_L1 File_Name=FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF Geolocation_Latitude_band_name=none Geolocation_Latitude_Description=Latitude of Every five Pixels in WGS84 Geolocation_Latitude_FillValue=65535 Geolocation_Latitude_Intercept=0 Geolocation_Latitude_Line_number=0,5,10..... Geolocation_Latitude_long_name=Latitude for Every five Pixels Geolocation_Latitude_Pixel_number=0,5,10..... Geolocation_Latitude_Slope=1 Geolocation_Latitude_units=degree Geolocation_Latitude_valid_range=-90 90 Geolocation_Longitude_band_name=none Geolocation_Longitude_Description=Longitude of Every five Pixels in WGS84 Geolocation_Longitude_FillValue=65535 Geolocation_Longitude_Intercept=0 Geolocation_Longitude_Line_number=0,5,10..... Geolocation_Longitude_long_name=Longitude for Every five Pixels Geolocation_Longitude_Pixel_number=0,5,10..... Geolocation_Longitude_Slope=1 Geolocation_Longitude_units=degree Geolocation_Longitude_valid_range=-180 180 MeanAnomaly=49927490 MeanMotion=103454 Number_Of_Day_mode_scans=200 Number_of_Night_mode_scans=0 Number_Of_Scans=200 Observing_Beginning_Date=2025-06-05 Observing_Beginning_Time=09:05:00.989 Observing_Ending_Date=2025-06-05 Observing_Ending_Time=09:09:59.491 OrbitalInclination=27524791 Orbit_Direction=A Orbit_Number=39153 Orbit_Period(min.)=102 Orbit_Point_Latitude=26.835192 22.534163 44.272575 39.032059 Orbit_Point_Longitude=99.692139 72.056732 97.902412 64.161842 PerigeeArgument=20222223 Pixels_per_Scan=2048 QA_QA_Frame_Flag_band_name=none QA_QA_Frame_Flag_Description=Quality Assurance Flag for Each Channel.Note: Quality Assurance for Each channel is designed for the 32-bit binary code, and each 0 or 1 indicates good or bad quality. As described below,Bit0~Bit24 indicate the data quality for each channel, =0, good; =1, bad. The criterion for good quality is that the data is integrated and within dynamic range.Bit25~Bit31, reserved ,and default as 0 QA_QA_Frame_Flag_FillValue= QA_QA_Frame_Flag_Intercept=0 QA_QA_Frame_Flag_long_name= Quality Assurance Flag for Each Channel QA_QA_Frame_Flag_Slope=1 QA_QA_Frame_Flag_units=none QA_QA_Frame_Flag_valid_range= Reference_Ellipsoid_Model_ID=WGS84 Responser=NSMC Satellite_Name=FY-3D Scan_Frame_number=200 Scan_Line_number=2000 Sensor_Identification_Code=MERSI Sensor_Name=Medium Resolution Spectral Imager Software_Revision_Date= Solar_Irradiance=2017.963 1828.387 1554.807 952.49353 363.07849 232.41879 97.017998 1700.734 1903.334 1968.184 1830.053 1504.9139 1399.233 1277.788 955.24152 884.80988 828.42151 820.49359 680.8728 Successfully_pre-pressed_Scans=199 SV_Count_Contaminated_Scans=0 TBB_Trans_Coefficient_A=1.00103 1.00085 1.00125 1.0003 1.00133 1.00065 TBB_Trans_Coefficient_B=-0.47589999 -0.31389999 -0.26620001 -0.0513 -0.073399998 0.087499999 Version_Of_Calibration_Parameter=V 1.0.1 Version_Of_Software= Subdatasets: SUBDATASET_1_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Calibration/BB_DN_average SUBDATASET_1_DESC=[25x200] //Calibration/BB_DN_average (32-bit floating-point) SUBDATASET_2_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Calibration/IR_Cal_Coeff SUBDATASET_2_DESC=[6x4x200] //Calibration/IR_Cal_Coeff (32-bit floating-point) SUBDATASET_3_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Calibration/SV_DN_average SUBDATASET_3_DESC=[25x200] //Calibration/SV_DN_average (32-bit floating-point) SUBDATASET_4_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Calibration/VIS_Cal_Coeff SUBDATASET_4_DESC=[19x3] //Calibration/VIS_Cal_Coeff (32-bit floating-point) SUBDATASET_5_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Calibration/VOC_DN_average SUBDATASET_5_DESC=[25x200] //Calibration/VOC_DN_average (32-bit floating-point) SUBDATASET_6_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Data/EV_1KM_Emissive SUBDATASET_6_DESC=[4x2000x2048] //Data/EV_1KM_Emissive (16-bit unsigned integer) SUBDATASET_7_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Data/EV_1KM_RefSB SUBDATASET_7_DESC=[15x2000x2048] //Data/EV_1KM_RefSB (16-bit unsigned integer) SUBDATASET_8_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Data/EV_250_Aggr.1KM_Emissive SUBDATASET_8_DESC=[2x2000x2048] //Data/EV_250_Aggr.1KM_Emissive (16-bit unsigned integer) SUBDATASET_9_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Data/EV_250_Aggr.1KM_RefSB SUBDATASET_9_DESC=[4x2000x2048] //Data/EV_250_Aggr.1KM_RefSB (16-bit unsigned integer) SUBDATASET_10_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Geolocation/Latitude SUBDATASET_10_DESC=[400x410] //Geolocation/Latitude (32-bit floating-point) SUBDATASET_11_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Geolocation/Longitude SUBDATASET_11_DESC=[400x410] //Geolocation/Longitude (32-bit floating-point) SUBDATASET_12_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://QA/QA_Frame_Flag SUBDATASET_12_DESC=[200x1] //QA/QA_Frame_Flag (64-bit integer) Corner Coordinates: Upper Left ( 0.0, 0.0) Lower Left ( 0.0, 512.0) Upper Right ( 512.0, 0.0) Lower Right ( 512.0, 512.0) Center ( 256.0, 256.0) Driver: HDF5/Hierarchical Data Format Release 5 Files: D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_GEO1K_MS.HDF Size is 512, 512 Metadata: AdditionalAnnotation=MERSI Geolocation use GPS data. AscendingNodeLongitude=33179878 Calibration_Parameter_Revision_Date=2017-11-25 Count_CaliErr_Scans=0 Count_GeolErr_Scans=4 Dataset_Name=Global MERSI Data Data_Creating_Date=2025-06-05 Data_Creating_Time=11:35:31.000 Data_Integrity=0 Day_Or_Night_Flag=D EarthSun_Distance_Ratio=1.01457519141879 Eccentricity=267488 EpochTime=250605 File_Alias_Name=MERSI_1KM_L1 File_Name=FY3D_MERSI_GBAL_L1_20250605_0905_GEO1K_MS.HDF Geolocation_DEM_band_name=none Geolocation_DEM_Description=The elevation data based on Digital Elevation Model Geolocation_DEM_FillValue=-32767 Geolocation_DEM_Intercept=0 Geolocation_DEM_long_name=Digital Elevation Model Geolocation_DEM_Slope=1 Geolocation_DEM_units=meter Geolocation_DEM_valid_range=-400 10000 Geolocation_LandCover_band_name=none Geolocation_LandCover_Description=The type of land cover, 0 Water 1 Evergreen Needleleaf Forest 2 Evergreen Broadleaf Forest 3 Deciduous Needleleaf Forest 4 Deciduous Broadleaf Forest 5 Mixed Forests 6 Closed Shrublands 7 Open Shrublands 8 Woody Savannas 9 Savannas 10 Grasslands11 Permanent Wetlands 12 Croplands 13 Urban and Built-Up 14 Cropland/Natural Vegetation Mosaic15 Snow and Ice 16 Barren or Sparsely Vegetated17 (IGBP Water Bodies, recoded to 0 for MODIS Land Product consistency.)254 Unclassified255 Fill Value Geolocation_LandCover_FillValue=255 Geolocation_LandCover_Intercept=0 Geolocation_LandCover_long_name=Land Cover Geolocation_LandCover_Slope=1 Geolocation_LandCover_units=none Geolocation_LandCover_valid_range=0 254 Geolocation_LandSeaMask_band_name=none Geolocation_LandSeaMask_Description=The type of earth surface:0 = Shallow Ocean (Ocean < 5km from coast or < 50m deep).1 = Land (not anything else).2 = Ocean Coastlines and Lake Shorelines.3 = Shallow Inland Water (Inland Water < 5km from shore or < 50m deep).4 = Ephemeral (intermittent) Water.5 = Deep Inland Water ( Inland water > 5km from shoreline and > 50m deep).6 = Moderate or Continental Ocean (Ocean > 5km from coast and > 50m deep and <500m deep).7 = Deep Ocean (Ocean > 500m deep). Geolocation_LandSeaMask_FillValue=255 Geolocation_LandSeaMask_Intercept=0 Geolocation_LandSeaMask_long_name=Land Sea Mask Geolocation_LandSeaMask_Slope=1 Geolocation_LandSeaMask_units=none Geolocation_LandSeaMask_valid_range=0 7 Geolocation_Latitude_band_name=none Geolocation_Latitude_Description=Latitude of each pixel in Earth Topography based on WGS84 and Digital Elevation Model Geolocation_Latitude_FillValue=65535 Geolocation_Latitude_Intercept=0 Geolocation_Latitude_long_name=Latitude Geolocation_Latitude_Slope=1 Geolocation_Latitude_units=degree Geolocation_Latitude_valid_range=-90 90 Geolocation_Longitude_band_name=none Geolocation_Longitude_Description=Longitude of each pixel in Earth Topography based on WGS84 and Digital Elevation Model Geolocation_Longitude_FillValue=65535 Geolocation_Longitude_Intercept=0 Geolocation_Longitude_long_name=Longitude Geolocation_Longitude_Slope=1 Geolocation_Longitude_units=degree Geolocation_Longitude_valid_range=-180 180 Geolocation_SensorAzimuth_band_name=none Geolocation_SensorAzimuth_Description=Sensor azimuth angle at the geolocated beam position center Geolocation_SensorAzimuth_FillValue=65535 Geolocation_SensorAzimuth_Intercept=0 Geolocation_SensorAzimuth_long_name=Sensor Azimuth Geolocation_SensorAzimuth_Slope=0.0099999998 Geolocation_SensorAzimuth_units=degree Geolocation_SensorAzimuth_valid_range=0 36000 Geolocation_SensorZenith_band_name=none Geolocation_SensorZenith_Description=Sensor zenith angle at the geolocated beam position center Geolocation_SensorZenith_FillValue=-32767 Geolocation_SensorZenith_Intercept=0 Geolocation_SensorZenith_long_name=Sensor Zenith Geolocation_SensorZenith_Slope=0.0099999998 Geolocation_SensorZenith_units=degree Geolocation_SensorZenith_valid_range=0 18000 Geolocation_SolarAzimuth_band_name=none Geolocation_SolarAzimuth_Description=Solar azimuth angle at the geolocated beam position center Geolocation_SolarAzimuth_FillValue=65535 Geolocation_SolarAzimuth_Intercept=0 Geolocation_SolarAzimuth_long_name=Solar Azimith Geolocation_SolarAzimuth_Slope=0.0099999998 Geolocation_SolarAzimuth_units=degree Geolocation_SolarAzimuth_valid_range=0 36000 Geolocation_SolarZenith_band_name=none Geolocation_SolarZenith_Description=Solar zenith angle at the geolocated beam position center Geolocation_SolarZenith_FillValue=-32767 Geolocation_SolarZenith_Intercept=0 Geolocation_SolarZenith_long_name=Solar Zenith Geolocation_SolarZenith_Slope=0.0099999998 Geolocation_SolarZenith_units=degree Geolocation_SolarZenith_valid_range=0 18000 MeanAnomaly=49927490 MeanMotion=103454 Number_Of_Day_mode_scans=200 Number_of_Night_mode_scans=0 Number_Of_Scans=200 Observing_Beginning_Date=2025-06-05 Observing_Beginning_Time=09:05:00.989 Observing_Ending_Date=2025-06-05 Observing_Ending_Time=09:09:59.491 OrbitalInclination=27524791 Orbit_Direction=A Orbit_Number=39153 Orbit_Period(min.)=102 Orbit_Point_Latitude=26.835192 22.534163 44.272575 39.032059 Orbit_Point_Longitude=99.692139 72.056732 97.902412 64.161842 PerigeeArgument=20222223 Reference_Ellipsoid_Model_ID=WGS84 Responser=NSMC Satellite_Name=FY-3D Sensor_Identification_Code=MERSI Sensor_Name=Medium Resolution Spectral Imager Software_Revision_Date=2017-11-25 Successfully_pre-pressed_Scans=199 Temporary_QA_index_band_name= Temporary_QA_index_FillValue=-1 Temporary_QA_index_Intercept=0 Temporary_QA_index_long_name=Quality Assurance_index Temporary_QA_index_Slope=1 Temporary_QA_index_units=none Temporary_QA_index_valid_range=0 50000 Timedata_DayNightFlag_band_name=none Timedata_DayNightFlag_Description=The flag for indicating the pixels in the scan line for Day(0) Night(1) or Mix(2) Timedata_DayNightFlag_FillValue=255 Timedata_DayNightFlag_Intercept=0 Timedata_DayNightFlag_long_name=Nadir Day(0) Night(1) or Mix(2) Flag Timedata_DayNightFlag_Slope=1 Timedata_DayNightFlag_units=none Timedata_DayNightFlag_valid_range=0 2 Timedata_Day_Count_band_name=none Timedata_Day_Count_Description=Day Counted from 12:00am in Jan1, 2000 for the beginning time of observation in each scanline Timedata_Day_Count_FillValue=65535 Timedata_Day_Count_Intercept=0 Timedata_Day_Count_long_name=Day Count Since J2000.0 Timedata_Day_Count_Slope=1 Timedata_Day_Count_units=day Timedata_Day_Count_valid_range=6100 13200 Timedata_Millisecond_Count_band_name=none Timedata_Millisecond_Count_Description=Milliseconds counted from 12:00 am of each day for the beginning time of earth observation in each scanline Timedata_Millisecond_Count_FillValue=999999999 Timedata_Millisecond_Count_Intercept=0 Timedata_Millisecond_Count_long_name=Millisecond Count In a Day Timedata_Millisecond_Count_Slope=1 Timedata_Millisecond_Count_units=Millisecond Timedata_Millisecond_Count_valid_range=0 86400000 Version_Of_Calibration_Parameter=V 1.0.1 Version_Of_Software=V 1.0.1 Subdatasets: SUBDATASET_1_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_GEO1K_MS.HDF"://Geolocation/DEM SUBDATASET_1_DESC=[2000x2048] //Geolocation/DEM (16-bit integer) SUBDATASET_2_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_GEO1K_MS.HDF"://Geolocation/LandCover SUBDATASET_2_DESC=[2000x2048] //Geolocation/LandCover (8-bit unsigned character) SUBDATASET_3_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_GEO1K_MS.HDF"://Geolocation/LandSeaMask SUBDATASET_3_DESC=[2000x2048] //Geolocation/LandSeaMask (8-bit unsigned character) SUBDATASET_4_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_GEO1K_MS.HDF"://Geolocation/Latitude SUBDATASET_4_DESC=[2000x2048] //Geolocation/Latitude (32-bit floating-point) SUBDATASET_5_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_GEO1K_MS.HDF"://Geolocation/Longitude SUBDATASET_5_DESC=[2000x2048] //Geolocation/Longitude (32-bit floating-point) SUBDATASET_6_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_GEO1K_MS.HDF"://Geolocation/SensorAzimuth SUBDATASET_6_DESC=[2000x2048] //Geolocation/SensorAzimuth (16-bit unsigned integer) SUBDATASET_7_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_GEO1K_MS.HDF"://Geolocation/SensorZenith SUBDATASET_7_DESC=[2000x2048] //Geolocation/SensorZenith (16-bit integer) SUBDATASET_8_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_GEO1K_MS.HDF"://Geolocation/SolarAzimuth SUBDATASET_8_DESC=[2000x2048] //Geolocation/SolarAzimuth (16-bit unsigned integer) SUBDATASET_9_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_GEO1K_MS.HDF"://Geolocation/SolarZenith SUBDATASET_9_DESC=[2000x2048] //Geolocation/SolarZenith (16-bit integer) SUBDATASET_10_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_GEO1K_MS.HDF"://Temporary/QA_index SUBDATASET_10_DESC=[2000x1] //Temporary/QA_index (32-bit unsigned integer) SUBDATASET_11_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_GEO1K_MS.HDF"://Timedata/DayNightFlag SUBDATASET_11_DESC=[200x1] //Timedata/DayNightFlag (8-bit unsigned character) SUBDATASET_12_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_GEO1K_MS.HDF"://Timedata/Day_Count SUBDATASET_12_DESC=[200x1] //Timedata/Day_Count (32-bit integer) SUBDATASET_13_NAME=HDF5:"D:\FYData\Data\Q202506121354284980\FY3D_MERSI_GBAL_L1_20250605_0905_GEO1K_MS.HDF"://Timedata/Millisecond_Count SUBDATASET_13_DESC=[200x1] //Timedata/Millisecond_Count (32-bit integer) Corner Coordinates: Upper Left ( 0.0, 0.0) Lower Left ( 0.0, 512.0) Upper Right ( 512.0, 0.0) Lower Right ( 512.0, 512.0) Center ( 256.0, 256.0) [('HDF5:"D:\\FYData\\Data\\Q202506121354284980\\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Calibration/BB_DN_average', '[25x200] //Calibration/BB_DN_average (32-bit floating-point)'), ('HDF5:"D:\\FYData\\Data\\Q202506121354284980\\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Calibration/IR_Cal_Coeff', '[6x4x200] //Calibration/IR_Cal_Coeff (32-bit floating-point)'), ('HDF5:"D:\\FYData\\Data\\Q202506121354284980\\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Calibration/SV_DN_average', '[25x200] //Calibration/SV_DN_average (32-bit floating-point)'), ('HDF5:"D:\\FYData\\Data\\Q202506121354284980\\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Calibration/VIS_Cal_Coeff', '[19x3] //Calibration/VIS_Cal_Coeff (32-bit floating-point)'), ('HDF5:"D:\\FYData\\Data\\Q202506121354284980\\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Calibration/VOC_DN_average', '[25x200] //Calibration/VOC_DN_average (32-bit floating-point)'), ('HDF5:"D:\\FYData\\Data\\Q202506121354284980\\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Data/EV_1KM_Emissive', '[4x2000x2048] //Data/EV_1KM_Emissive (16-bit unsigned integer)'), ('HDF5:"D:\\FYData\\Data\\Q202506121354284980\\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Data/EV_1KM_RefSB', '[15x2000x2048] //Data/EV_1KM_RefSB (16-bit unsigned integer)'), ('HDF5:"D:\\FYData\\Data\\Q202506121354284980\\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Data/EV_250_Aggr.1KM_Emissive', '[2x2000x2048] //Data/EV_250_Aggr.1KM_Emissive (16-bit unsigned integer)'), ('HDF5:"D:\\FYData\\Data\\Q202506121354284980\\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Data/EV_250_Aggr.1KM_RefSB', '[4x2000x2048] //Data/EV_250_Aggr.1KM_RefSB (16-bit unsigned integer)'), ('HDF5:"D:\\FYData\\Data\\Q202506121354284980\\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Geolocation/Latitude', '[400x410] //Geolocation/Latitude (32-bit floating-point)'), ('HDF5:"D:\\FYData\\Data\\Q202506121354284980\\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://Geolocation/Longitude', '[400x410] //Geolocation/Longitude (32-bit floating-point)'), ('HDF5:"D:\\FYData\\Data\\Q202506121354284980\\FY3D_MERSI_GBAL_L1_20250605_0905_1000M_MS.HDF"://QA/QA_Frame_Flag', '[200x1] //QA/QA_Frame_Flag (64-bit integer)')]
06-14
We are given a log file that contains a sequence of events related to an update process on a device. The log is from the perspective of an automated test system (autotest) that is monitoring the update process. The key events in the log: 1. The test system is trying to establish an SSH connection to the device at 192.168.231.225. Initially, the connection is down, so it tries to start a new connection. 2. At 04:38:48, there's a timeout error waiting for the main SSH connection to come up. 3. Then, at 04:39:06, we see that the device has switched slots (likely due to an update) and the test breaks out of a waiting loop. 4. The test then checks the status of the update engine by running `update_engine_client --status` via SSH. The output shows that the update is in the DOWNLOADING state with progress at about 13%. 5. The test then retrieves the update_engine log and we see the log entries from the device's update_engine daemon. The update_engine log shows: - The update_engine starts at 04:39:01. - It identifies that the device is booted from slot A (slot 0). - It's an unofficial build, so periodic update checks are blocked. - It begins an update for a DLC (Downloadable Content) called "timeofday-dlc". The download URL is provided, and the download completes successfully (HTTP 200) at 04:39:05, with 23064576 bytes downloaded. - After verifying the hash, the DLC is mirrored and set to Active. - Then, the update_engine starts another update for a different DLC: "ml-dlc-a97333ed-3157-49a3-b503-2d2d3f23c81d". The log shows the installation starting for this DLC. However, the test then fails because it cannot find the update_engine logs to verify the update. The last lines in the log are: ``` 06/05 04:39:09.952 DEBUG| test:0439| Test failed due to update_engine logs are missing. Cannot verify update.. Exception log follows the after_iteration_hooks. ``` This indicates that the test expected to find update_engine logs for the entire update process, but at the time it checked (04:39:09), the log for the second DLC update was not available (or the test looked in the wrong place). But note: the test did retrieve the update_engine log at 04:39:09.856 (the file `update_engine.20250604-213901`). However, that log only covers the first DLC (timeofday-dlc) and the beginning of the second DLC (ml-dlc). The second DLC update was still in progress when the test checked. The test then, at 04:39:09.920, runs `ls -t -1 /var/log/update_engine/` and gets only one log file: `update_engine.20250604-213901`. This is the same log that was previously retrieved. The test then fails because it expects to see logs for the entire update (which might include multiple log files or a more recent one) but only one is present. It's possible that the update_engine daemon rotates logs and the test expects a new log for the second update? However, the log we see spans from 04:39:01 to 04:39:07 (and beyond, but the test only retrieved up to 04:39:09.913 in the log). The test failure message says "update_engine logs are missing", meaning it couldn't find the logs it expected. Alternatively, the test might be looking for a log entry that confirms the entire update (including the second DLC) completed, but the log we have only shows the start of the second DLC update. Given the information, the problem appears to be that the test is failing because it cannot find the expected update_engine logs to verify the update. This could be because: - The test is not waiting long enough for the update to complete and the logs to be written. - The update_engine log rotation might have occurred and the test is not accounting for the latest log file. - The test expects a specific log entry that is not present in the retrieved log. However, from the provided log, we can see that the update_engine was in the middle of updating the second DLC when the test checked. The test then immediately tried to get the logs again and found only one log file, which it had already retrieved. The test code seems to be in a state where it breaks out of the waiting loop because it detected a slot switch (which happened for the first DLC) but then it tries to verify the entire update (which includes multiple DLCs) and fails because the second DLC update was still ongoing and the logs were not complete. In summary, the issue is that the test did not wait for the entire update (all DLCs) to complete before attempting to verify the logs. It only waited for the first DLC to complete and the slot switch, but then proceeded to check the logs while the second DLC update was still in progress. Recommendation: The test should wait until the update_engine status indicates that the update is fully completed (e.g., IDLE) and then verify the logs. 翻译成中文
06-06
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值