[代码] 如何处理添加和删除文档请求

who got updateRequest first?

the answer is SolrDispatchFilter. let's look a graph first:

1.  .doFilter receive any update request first.  doFilter will choice a right core for this request.

      a. if the request specify a core(for index request) or cores(for search request), give the request to the specified core(s)

      b. if not, we will try the collection. find the first leader core then send this request to it. usually the

    then doFilter will find the rigth requestHander for the request according the request. the requestHandler would be  :

  

the REST API decide which handler will be taken

2. now, we got a handler and a core to process this request. bring the request to core.execute(handler, req, rsp). the specified core will deal with the rest.

the core add some header to response the call supper.handleRequestBody(req, rsp)

  

above is the full code of HandleRequestBody(……)

1. create all the processor in the processorChain and return the first processor(LogUpdateProcessor)

 the processorChain : 

2. got a documentLoader according to the requestHandler.  then call Loader.load(req, rsp, stream)

 .load() call processUpdate and process different commands respectively.   all update request pass to processor created in the step 1. the processor is LogUpdateProcessor. it call next processor in the processor chain  recursively

 the commit command with add or delete request will be removed in this progress. commit will execute at the next step 3

 if we recieve a json format index request, the documentLoader here would be JsonLoader.  we discuss add doc request processing which is the same as delete request

 we look into the JsonLoader.processUpdate()

void processUpdate() throws IOException     {       int ev = parser.nextEvent();       while( ev != JSONParser.EOF ) {           switch( ev )         {           case JSONParser.ARRAY_START:             handleAdds();             break;           case JSONParser.STRING:           if( parser.wasKey() ) {             String v = parser.getString();             if( v.equals( UpdateRequestHandler.ADD ) ) {               int ev2 = parser.nextEvent();               if (ev2 == JSONParser.OBJECT_START) {                 processor.processAdd( parseAdd() );               } else if (ev2 == JSONParser.ARRAY_START) {                 handleAdds();               } else {                 assertEvent(ev2, JSONParser.OBJECT_START);               }             }             else if( v.equals( UpdateRequestHandler.COMMIT ) ) {               CommitUpdateCommand cmd = new CommitUpdateCommand(req,  false );               cmd.waitSearcher = true;               parseCommitOptions( cmd );               processor.processCommit( cmd );             }             else if( v.equals( UpdateRequestHandler.OPTIMIZE ) ) {               CommitUpdateCommand cmd = new CommitUpdateCommand(req, true );               cmd.waitSearcher = true;               parseCommitOptions( cmd );               processor.processCommit( cmd );             }             else if( v.equals( UpdateRequestHandler.DELETE ) ) {               handleDeleteCommand();             }             else if( v.equals( UpdateRequestHandler.ROLLBACK ) ) {               processor.processRollback( parseRollback() );             }             else {               throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Unknown command: "+v+" ["+parser.getPosition()+"]" );             }             break;           }           // fall through           case JSONParser.LONG:         case JSONParser.NUMBER:         case JSONParser.BIGNUMBER:         case JSONParser.BOOLEAN:         case JSONParser.NULL:           log.info( "can't have a value here! "               +JSONParser.getEventString(ev)+" "+parser.getPosition() );           case JSONParser.OBJECT_START:         case JSONParser.OBJECT_END:         case JSONParser.ARRAY_END:           break;           default:           log.info("Noggit UNKNOWN_EVENT_ID:"+ev);           break;         }         // read the next event         ev = parser.nextEvent();       }     }

so for ADD, processor.processAdd(cmd) is called. here, processor is LogUpdateProcessor.

LogUpdateProcessor.processAdd(cmd)

public void processAdd(AddUpdateCommand cmd) throws IOException {     if (logDebug) { log.debug("PRE_UPDATE " + cmd.toString()); }       // call delegate first so we can log things like the version that get set later     if (next != null) next.processAdd(cmd);       // Add a list of added id's to the response     if (adds == null) {       adds = new ArrayList<String>();       toLog.add("add",adds);     }       if (adds.size() < maxNumToLog) {//10       long version = cmd.getVersion();       String msg = cmd.getPrintableId();       if (version != 0) msg = msg + " (" + version + ')';       adds.add(msg);     }       numAdds++;   }

it call next.processAdd(cmd), DistributedUpdateProcessor.processAdd(cmd), firstly . then LogUpdateProcessor.processAdd( cmd) will write some logs info.

now, the request  deal by DistributedUpdateProcessor.processAdd(cmd).  DistributedUpdateProcessor is the key point for cloud solr. it decid which silce the request should send to.

follow code hash the request  cmd then decid which nodes this request should sennd to.  whether or not  i'm a leader. seting forwardToLeader flag to indicate whether or not to distribute to leader.

int hash = 0;     if (zkEnabled) {       zkCheck();       hash = hash(cmd);       nodes = setupRequest(hash);     } else {       isLeader = getNonZkLeaderAssumption(req);     }

if (!forwardToLeader) {       // clone the original doc       SolrInputDocument clonedDoc = cmd.solrDoc.deepCopy();       dropCmd = versionAdd(cmd, clonedDoc);       cmd.solrDoc = clonedDoc;     }

if i'm leader, call versionAdd(cmd, cloneDoc). this function will call local methed to add this index request

if there are any node need to distribute to then do :

if (nodes != null) {       params = new ModifiableSolrParams(req.getParams());       params.set(DISTRIB_UPDATE_PARAM,                  (isLeader ?                   DistribPhase.FROMLEADER.toString() :                   DistribPhase.TOLEADER.toString()));       params.remove("commit"); // this will be distributed from the local commit       cmdDistrib.distribAdd(cmd, nodes, params);  //key function for distribute command     }

cmdDistrib.distribAdd(cmd, nodes, params) will send this addRequest(cmd) to all nodes.

3. process commit or rollback if the request contain those kind of  commands

4. call processor.finish() to finish the request processing and finish the severlet response