protobuf学习笔记-官方示例

示例简介

本例是一个可以读写联系信息的“地址本”应用，每个人在地址本里有个id、名字、邮箱地址和联系电话。

---

编译器安装

• window
  
  1. 下载protocol compiler
  2. 将下载到的可执行文件放置到相应目录，并配置到环境变量path中
• linux
  略

---

定义protocol格式

为了创建地址本应用，我们需要开始定义一个.proto文件。.proto文件定义很简单：为需要序列化的数据都添加一个message，然后指定message中字段指定类型和名称。具体定义如下所示：

package tutorial;

option java_package = "com.example.tutorial";
option java_outer_classname = "AddressBookProtos";

message Person {
  required string name = 1;
  required int32 id = 2;
  optional string email = 3;

  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }

  message PhoneNumber {
    required string number = 1;
    optional PhoneType type = 2 [default = HOME];
  }

  repeated PhoneNumber phone = 4;
}

message AddressBook {
  repeated Person person = 1;
}

如你所见，上面的语法与C++和Java非常相似。下面我们一步步分析该文件：

• package：避免不同的项目命名冲突

• java_package和java_outer_classname

  • java_package：指定生成的类会在哪个包下，如果不显示指定，使用package的声明
  • java_outer_classname：指定.proto文件包含的所有类的外部类名称，如果不显示指定，使用.ptoto文件名称的驼峰形式作为类名

• message：一个message是一系列字段的集合。

• field
  
  • type：bool、int32、float、string、enum、message
  • modifier：
    
    • required：require修饰的字段值必须设置，否则在反序列话时会遇到RuntimeException，序列化这样的数据也会遇到IOException。
    • optional：optional修饰的字段值可有可无，对于简单数据类型，可以设置默认值。否则，系统自己设置默认值：数字类型为0，字符类型为空字符串，布尔值为false，
    • repeated：repeated修饰的字段值可能有0个或多个，并且protobuf保留了这些值的顺序
  • “=1, =2”：每个字段都有类似这样的指定，这个是用于标记每个元素在二进制编码中的标识。1-15比更多的数字可以少用一个byte，所以在优化的时候，可以给那些公共使用或者重复的字段指定1-15。

---

编译Protocol Buffer

在命令行使用protoc命令可以生成类来读写地址本，在命令行里执行protoc -I=$SRC_DIR --java_out=$DST_DIR $SRC_DIR/addressbook.proto，生成的com/example/tutorial/AddressBookProtos.java在指定的路径下。protoc的命令行用法如下所示：

Usage: protoc [OPTION] PROTO_FILES
Parse PROTO_FILES and generate output based on the options given:
-IPATH, --proto_path=PATH   Specify the directory in which to search for
                          imports.  May be specified multiple times;
                          directories will be searched in order.  If not
                          given, the current working directory is used.
--version                   Show version info and exit.
-h, --help                  Show this text and exit.
--encode=MESSAGE_TYPE       Read a text-format message of the given type
                          from standard input and write it in binary
                          to standard output.  The message type must
                          be defined in PROTO_FILES or their imports.
--decode=MESSAGE_TYPE       Read a binary message of the given type from
                          standard input and write it in text format
                          to standard output.  The message type must
                          be defined in PROTO_FILES or their imports.
--decode_raw                Read an arbitrary protocol message from
                          standard input and write the raw tag/value
                          pairs in text format to standard output.  No
                          PROTO_FILES should be given when using this
                          flag.
-oFILE,                     Writes a FileDescriptorSet (a protocol buffer,
--descriptor_set_out=FILE defined in descriptor.proto) containing all of
                          the input files to FILE.
--include_imports           When using --descriptor_set_out, also include
                          all dependencies of the input files in the
                          set, so that the set is self-contained.
--include_source_info       When using --descriptor_set_out, do not strip
                          SourceCodeInfo from the FileDescriptorProto.
                          This results in vastly larger descriptors that
                          include information about the original
                          location of each decl in the source file as
                          well as surrounding comments.
--error_format=FORMAT       Set the format in which to print errors.
                          FORMAT may be 'gcc' (the default) or 'msvs'
                          (Microsoft Visual Studio format).
--print_free_field_numbers  Print the free field numbers of the messages
                          defined in the given proto files. Groups share
                          the same field number space with the parent
                          message. Extension ranges are counted as
                          occupied fields numbers.
--plugin=EXECUTABLE         Specifies a plugin executable to use.
                          Normally, protoc searches the PATH for
                          plugins, but you may specify additional
                          executables not in the path using this flag.
                          Additionally, EXECUTABLE may be of the form
                          NAME=PATH, in which case the given plugin name
                          is mapped to the given executable even if
                          the executable's own name differs.
--cpp_out=OUT_DIR           Generate C++ header and source.
--java_out=OUT_DIR          Generate Java source file.
--python_out=OUT_DIR        Generate Python source file.

---

Protocol Buffer API简介

查看生成的代码发现：
1. AddressBookProtos.java定义了一个叫AddressBookProtos的类，AddressBookProtos类嵌套了addressbook.proto中声明的每一个message对应的类
2. 每一个message类有一个自己的Builder类，用来创建类实例
3. message类和builder类都有自动生成的字段访问方法，但是message只有getter方法，builder有getter和setter
Person类的getter代码：

    // required string name = 1;
    public boolean hasName();
    public String getName();

    // required int32 id = 2;
    public boolean hasId();
    public int getId();

    // optional string email = 3;
    public boolean hasEmail();
    public String getEmail();

    // repeated .tutorial.Person.PhoneNumber phone = 4;
    public List<PhoneNumber> getPhoneList();
    public int getPhoneCount();
    public PhoneNumber getPhone(int index);

Person.Builder的getter和setter代码

    // required string name = 1;
    public boolean hasName();
    public java.lang.String getName();
    public Builder setName(String value);
    public Builder clearName();

    // required int32 id = 2;
    public boolean hasId();
    public int getId();
    public Builder setId(int value);
    public Builder clearId();

    // optional string email = 3;
    public boolean hasEmail();
    public String getEmail();
    public Builder setEmail(String value);
    public Builder clearEmail();

    // repeated .tutorial.Person.PhoneNumber phone = 4;
    public List<PhoneNumber> getPhoneList();
    public int getPhoneCount();
    public PhoneNumber getPhone(int index);
    public Builder setPhone(int index, PhoneNumber value);
    public Builder addPhone(PhoneNumber value);
    public Builder addAllPhone(Iterable<PhoneNumber> value);
    public Builder clearPhone();

4. 每个字段都有个Java-Beans风格的getter和setter方法，单一值的字段还有个has方法，表示该字段是否有赋值。此外每个字段都有个clear方法，用于置空该字段。
5. 多值字段还有一些额外方法：获取值个数的count方法，通过下标获取和修改指定数组元素的getter和setter方法，增加新的元素的add方法，增加一个容器内全部元素的addAll方法
6. 访问方法都是使用驼峰式的命名方法，即使.proto文件使用带下划线的小写。为了针对所有语言有一个好的命名规范，在.proto文件通常使用带下划线的小写字母的命名方式。
7. message类实例：message类实例与String相似，都是不可变的。创建一个message类实例：

1. 实例化message类的builder类实例，使用newBuilder()方法
2. 通过builder类实例设置字段值
3. 调用builder类实例的build()方法

8. message类和build类的其他通用方法

1. isInitialized()：检查必填字段是否设置了值
2. toString()：返回可阅读的message内容，通常用于调试
3. mergeFrom(Message other)：合并other的内容到当前实例
4. clear()：重置所有字段为空值

9. 解析于序列化

- byte[] toByteArray()：序列化一个message，返回一个byte数组
-static Person parseFrom(byte[] data)：解析给定的byte数组为message
- void writeTo(OutputStream output)：序列化一个message，且写入指定的输出流
- static Person parseFrom(InputStream input)：读取并解析指定输入流的message

10. Protocol buffer类扩展：不可以通过继承的方式来给生成的类添加方法和属性，可以通过包装类来实现

---

写一个消息

import com.example.tutorial.AddressBookProtos.AddressBook;
import com.example.tutorial.AddressBookProtos.Person;
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.InputStreamReader;
import java.io.IOException;
import java.io.PrintStream;

class AddPerson {
  // This function fills in a Person message based on user input.
  static Person PromptForAddress(BufferedReader stdin,
                                 PrintStream stdout) throws IOException {
    Person.Builder person = Person.newBuilder();

    stdout.print("Enter person ID: ");
    person.setId(Integer.valueOf(stdin.readLine()));

    stdout.print("Enter name: ");
    person.setName(stdin.readLine());

    stdout.print("Enter email address (blank for none): ");
    String email = stdin.readLine();
    if (email.length() > 0) {
      person.setEmail(email);
    }

    while (true) {
      stdout.print("Enter a phone number (or leave blank to finish): ");
      String number = stdin.readLine();
      if (number.length() == 0) {
        break;
      }

      Person.PhoneNumber.Builder phoneNumber =
        Person.PhoneNumber.newBuilder().setNumber(number);

      stdout.print("Is this a mobile, home, or work phone? ");
      String type = stdin.readLine();
      if (type.equals("mobile")) {
        phoneNumber.setType(Person.PhoneType.MOBILE);
      } else if (type.equals("home")) {
        phoneNumber.setType(Person.PhoneType.HOME);
      } else if (type.equals("work")) {
        phoneNumber.setType(Person.PhoneType.WORK);
      } else {
        stdout.println("Unknown phone type.  Using default.");
      }

      person.addPhone(phoneNumber);
    }

    return person.build();
  }

  // Main function:  Reads the entire address book from a file,
  //   adds one person based on user input, then writes it back out to the same
  //   file.
  public static void main(String[] args) throws Exception {
    if (args.length != 1) {
      System.err.println("Usage:  AddPerson ADDRESS_BOOK_FILE");
      System.exit(-1);
    }

    AddressBook.Builder addressBook = AddressBook.newBuilder();

    // Read the existing address book.
    try {
      addressBook.mergeFrom(new FileInputStream(args[0]));
    } catch (FileNotFoundException e) {
      System.out.println(args[0] + ": File not found.  Creating a new file.");
    }

    // Add an address.
    addressBook.addPerson(
      PromptForAddress(new BufferedReader(new InputStreamReader(System.in)),
                       System.out));

    // Write the new address book back to disk.
    FileOutputStream output = new FileOutputStream(args[0]);
    addressBook.build().writeTo(output);
    output.close();
  }
}

---

读一个消息

import com.example.tutorial.AddressBookProtos.AddressBook;
import com.example.tutorial.AddressBookProtos.Person;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.PrintStream;

class ListPeople {
  // Iterates though all people in the AddressBook and prints info about them.
  static void Print(AddressBook addressBook) {
    for (Person person: addressBook.getPersonList()) {
      System.out.println("Person ID: " + person.getId());
      System.out.println("  Name: " + person.getName());
      if (person.hasEmail()) {
        System.out.println("  E-mail address: " + person.getEmail());
      }

      for (Person.PhoneNumber phoneNumber : person.getPhoneList()) {
        switch (phoneNumber.getType()) {
          case MOBILE:
            System.out.print("  Mobile phone #: ");
            break;
          case HOME:
            System.out.print("  Home phone #: ");
            break;
          case WORK:
            System.out.print("  Work phone #: ");
            break;
        }
        System.out.println(phoneNumber.getNumber());
      }
    }
  }

  // Main function:  Reads the entire address book from a file and prints all
  //   the information inside.
  public static void main(String[] args) throws Exception {
    if (args.length != 1) {
      System.err.println("Usage:  ListPeople ADDRESS_BOOK_FILE");
      System.exit(-1);
    }

    // Read the existing address book.
    AddressBook addressBook =
      AddressBook.parseFrom(new FileInputStream(args[0]));

    Print(addressBook);
  }
}

---

扩展

在发布使用protocol buffer的代码后，肯定还会对protocol buffer格式定义改进。但为了保持新版本的向后兼容性， 旧版本的向前兼容性，新版本必须遵循以下规则：
1. 不能修改存在字段的标识数字
2. 不能添加和删除必须字段
3. 可以删除可选和重复字段
4. 可以增加可选和删除字段，但必须使用新的标识数字

---

高级使用

• 参考api
• 反射：与XML/JSON的相互转换