mahout in action推荐系统阅读笔记（7）_reloadfromjdbcdatamodel-优快云博客

本文链接：https://blog.youkuaiyun.com/softwarehe/article/details/8650750

本文介绍ReloadFromJDBCDataModel类，该类通过将JDBC数据模型加载到内存中来提升性能。文章详细解释了类的属性、构造函数及关键方法如reload和exportWithPrefs的工作原理。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

本部分介绍ReloadFromJDBCDataModel

ReloadFromJDBCDataModel

这个类的目标是提高jdbc类型的data model性能

属性

  private static final Logger log = LoggerFactory.getLogger(ReloadFromJDBCDataModel.class);

  private DataModel delegateInMemory;
  private final JDBCDataModel delegate;
  private final RefreshHelper refreshHelper;

delegate是原来的jdbcdatamodel，而delegateInMemory是转换后的内存data model，recommender访问的是内存data model

构造函数

  public ReloadFromJDBCDataModel(JDBCDataModel delegate) throws TasteException {
    Preconditions.checkNotNull(delegate, "Delegate cannot be null");
    this.delegate = delegate;
    refreshHelper = new RefreshHelper(new Callable<Void>() {
      @Override
      public Void call() {
        reload();
        return null;  //To change body of implemented methods use File | Settings | File Templates.
      }
    });
    refreshHelper.addDependency(delegate);
    reload();
    if (delegateInMemory == null) {
      throw new TasteException("Failed to load data into memory");
    }
  }

这里调用了reload函数初始化delegateInMemory

reload函数

  private void reload() {
    try {
      // Load new in-memory representation,
      log.info("Loading new JDBC delegate data...");
      DataModel newDelegateInMemory =
          delegate.hasPreferenceValues()
          ? new GenericDataModel(delegate.exportWithPrefs())
          : new GenericBooleanPrefDataModel(delegate.exportWithIDsOnly());
      // and then swap to it.
      log.info("New data loaded.");
      delegateInMemory = newDelegateInMemory;
    } catch (TasteException te) {
      log.warn("Error while reloading JDBC delegate data", te);
      // But continue with whatever is loaded
    }
  }

可以看到，调用delegate的exportWithPrefs初始化内存data model（这里是GenericDataModel或GenericBooleanPrefDataModel）

exportWithPrefs

  @Override
  public FastByIDMap<PreferenceArray> exportWithPrefs() throws TasteException {
    log.debug("Exporting all data");

    Connection conn = null;
    Statement stmt = null;
    ResultSet rs = null;

    FastByIDMap<PreferenceArray> result = new FastByIDMap<PreferenceArray>();

    try {
      conn = dataSource.getConnection();
      stmt = conn.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
      stmt.setFetchDirection(ResultSet.FETCH_FORWARD);
      stmt.setFetchSize(getFetchSize());

      log.debug("Executing SQL query: {}", getAllUsersSQL);
      rs = stmt.executeQuery(getAllUsersSQL);

      Long currentUserID = null;
      List<Preference> currentPrefs = new ArrayList<Preference>();
      while (rs.next()) {
        long nextUserID = getLongColumn(rs, 1);
        if (currentUserID != null && !currentUserID.equals(nextUserID) && !currentPrefs.isEmpty()) {
          result.put(currentUserID, new GenericUserPreferenceArray(currentPrefs));
          currentPrefs.clear();
        }
        currentPrefs.add(buildPreference(rs));
        currentUserID = nextUserID;
      }
      if (!currentPrefs.isEmpty()) {
        result.put(currentUserID, new GenericUserPreferenceArray(currentPrefs));
      }

      return result;

    } catch (SQLException sqle) {
      log.warn("Exception while exporting all data", sqle);
      throw new TasteException(sqle);
    } finally {
      IOUtils.quietClose(rs, stmt, conn);

    }
  }

很简单，就是查询数据库，返回preferences

更新

更新时候调用refresh方法即可，不过根据mahout in action的说法，最好是从recommender调用refresh，它会调用dependency的refresh函数，这样所有refresh一次调用完毕，参数设为null即可