December 9th Thursday

I found five ways to communicate between Erlang program and external program.

 

The following, all they are listed.

 

1. Ports

 

 

Erlang Program

 

First of all communication between Erlang and C must be established by creating the port. The Erlang process which creates a port is said to
be the connected process of the port. All communication to and from the port should go via the connected process. If the connected process terminates, so will the port (and the external program, if it is written correctly).

The port is created using the BIF open_port/2 with {spawn,ExtPrg} as the first argument. The string ExtPrg is the name of the external program, including any command line arguments. The second argument is a list of options, in this case only {packet,2}. This option says that a two byte length indicator will be used to simplify the communication between C and Erlang. Adding the length indicator will be done automatically by the Erlang port, but must be done explicitly in the external C program.

The process is also set to trap exits which makes it possible to detect if the external program fails.

-module(complex1).
-export([start/1, init/1]).

start(ExtPrg) ->
  spawn(?MODULE, init, [ExtPrg]).

init(ExtPrg) ->
  register(complex, self()),
  process_flag(trap_exit, true),
  Port = open_port({spawn, ExtPrg}, [{packet, 2}]),
  loop(Port).

Now it is possible to implement complex1:foo/1 and complex1:bar/1. They both send a message to the complex process and receive the reply.

foo(X) ->

  call_port({foo, X}).
bar(Y) ->
  call_port({bar, Y}).

call_port(Msg) ->
  complex ! {call, self(), Msg},
  receive
    {complex, Result} ->
      Result
  end.

The complex process encodes the message into a sequence of bytes, sends it to the port, waits for a reply, decodes the reply and sends it
back to the caller.

loop(Port) ->
  receive
    {call, Caller, Msg} ->
      Port ! {self(), {command, encode(Msg)}},
      receive
    {Port, {data, Data}} ->
          Caller ! {complex, decode(Data)}
      end,
      loop(Port)
 end.

Assuming that both the arguments and the results from the C functions will be less than 256, a very simple encoding/decoding scheme is employed where foo is represented by the byte 1, bar is represented by 2, and the argument/result is represented by a single byte as well.

encode({foo, X}) -> [1, X];

encode({bar, Y}) -> [2, Y].

decode([Int]) -> Int.

The resulting Erlang program, including functionality for stopping the port and detecting port failures is shown below.

-module(complex1).
-export([start/1, stop/0, init/1]).
-export([foo/1, bar/1]).

start(ExtPrg) ->
    spawn(?MODULE, init, [ExtPrg]).
stop() ->
    complex ! stop.

foo(X) ->
    call_port({foo, X}).
bar(Y) ->
    call_port({bar, Y}).

call_port(Msg) ->
    complex ! {call, self(), Msg},
    receive
 {complex, Result} ->
     Result
    end.

init(ExtPrg) ->
    register(complex, self()),
    process_flag(trap_exit, true),
    Port = open_port({spawn, ExtPrg}, [{packet, 2}]),
    loop(Port).

loop(Port) ->
    receive
 {call, Caller, Msg} ->
     Port ! {self(), {command, encode(Msg)}},
     receive
  {Port, {data, Data}} ->
      Caller ! {complex, decode(Data)}
     end,
     loop(Port);
 stop ->
     Port ! {self(), close},
     receive
  {Port, closed} ->
      exit(normal)
     end;
 {'EXIT', Port, Reason} ->
     exit(port_terminated)
    end.

encode({foo, X}) -> [1, X];
encode({bar, Y}) -> [2, Y].

decode([Int]) -> Int.

 

C Program

 

On the C side, it is necessary to write functions for receiving and sending data with two byte length indicators from/to Erlang. By default, the C program should read from standard input (file descriptor 0) and write to standard output (file descriptor 1). Examples of such functions, read_cmd/1 and write_cmd/2, are shown below.

/* erl_comm.c */

typedef unsigned char byte;

read_cmd(byte *buf)
{
  int len;

  if (read_exact(buf, 2) != 2)
    return(-1);
  len = (buf[0] << 8) | buf[1];
  return read_exact(buf, len);
}

write_cmd(byte *buf, int len)
{
  byte li;

  li = (len >> 8) & 0xff;
  write_exact(&li, 1);
 
  li = len & 0xff;
  write_exact(&li, 1);

  return write_exact(buf, len);
}

read_exact(byte *buf, int len)
{
  int i, got=0;

  do {
    if ((i = read(0, buf+got, len-got)) <= 0)
      return(i);
    got += i;
  } while (got<len);

  return(len);
}

write_exact(byte *buf, int len)
{
  int i, wrote = 0;

  do {
    if ((i = write(1, buf+wrote, len-wrote)) <= 0)
      return (i);
    wrote += i;
  } while (wrote<len);

  return (len);
}

Note that stdin and stdout are for buffered input/output and should not be used for the communication with Erlang! In the main function, the C program should listen for a message from Erlang and, according to the selected encoding/decoding scheme, use the first byte to determine which function to call and the second byte as argument to the function. The result of calling the function should then be sent
back to Erlang.

/* port.c */

typedef unsigned char byte;

int main() {
  int fn, arg, res;
  byte buf[100];

  while (read_cmd(buf) > 0) {
    fn = buf[0];
    arg = buf[1];
   
    if (fn == 1) {
      res = foo(arg);
    } else if (fn == 2) {
      res = bar(arg);
    }

    buf[0] = res;
    write_cmd(buf, 1);
  }
}

Note that the C program is in a while-loop checking for the return value of read_cmd/1. The reason for this is that the C program must detect when the port gets closed and terminate.

1. Compile the C code.

unix> gcc -o extprg complex.c erl_comm.c port.c

2. Start Erlang and compile the Erlang code.

unix> erl
Erlang (BEAM) emulator version 4.9.1.2

Eshell V4.9.1.2 (abort with ^G)
1> c(complex1).
{ok,complex1}

3. Run the example.

2> complex1:start("extprg").
<0.34.0>
3> complex1:foo(3).
4
4> complex1:bar(5).
10
5> complex1:stop().
stop

2. Erl_Interface

Erlang Program
The example below shows an Erlang program communicating with a C program over a plain port with home made encoding.

-module(complex1).

-export([start/1, stop/0, init/1]).

-export([foo/1, bar/1]).

start(ExtPrg) ->
   spawn(?MODULE, init, [ExtPrg]).

stop() ->
   complex ! stop.

foo(X) ->
   call_port({foo, X}).

bar(Y) ->
  call_port({bar, Y}).

call_port(Msg) ->
  complex ! {call, self(), Msg},
  receive
    {complex, Result} ->
        Result
  end.

init(ExtPrg) ->
  register(complex, self()),
  process_flag(trap_exit, true),

  Port = open_port({spawn, ExtPrg},
       [{packet, 2}]),
  loop(Port).

loop(Port) ->
  receive
    {call, Caller, Msg} ->
        Port ! {self(), {command, encode(Msg)}},
        receive {Port, {data, Data}} ->
           Caller ! {complex, decode(Data)}
        end,
        loop(Port);
    stop ->
      Port ! {self(), close},
      receive
        {Port, closed} ->
           exit(normal)
      end;
    {'EXIT', Port, Reason} ->
        exit(port_terminated)
end.

encode({foo, X}) ->
    [1, X];

encode({bar, Y}) ->
    [2, Y].

decode([Int]) -> Int.

Compared to the Erlang module above used for the plain port, there are two differences when using Erl_Interface on the C side: Since Erl_Interface operates on the Erlang external term format the port must be set to use binaries and, instead of inventing an encoding/decoding scheme, the BIFs term_to_binary/1 and binary_to_term/1 should be used. That is:

open_port({spawn, ExtPrg}, [{packet, 2}])

is replaced with:open_port({spawn, ExtPrg}, [{packet, 2}, binary])

And:

Port ! {self(), {command, encode(Msg)}},
receive
  {Port, {data, Data}} ->
    Caller ! {complex, decode(Data)}
end

is replaced with:

Port ! {self(), {command, term_to_binary(Msg)}},
receive
  {Port, {data, Data}} ->
    Caller ! {complex, binary_to_term(Data)}
end

The resulting Erlang program is shown below.
-module(complex2).
-export([start/1, stop/0, init/1]).
-export([foo/1, bar/1]).
start(ExtPrg) ->
    spawn(?MODULE, init, [ExtPrg]).
stop() ->
    complex ! stop.
foo(X) ->
    call_port({foo, X}).
bar(Y) ->
    call_port({bar, Y}).
call_port(Msg) ->
    complex ! {call, self(), Msg},
    receive
 {complex, Result} ->
     Result
    end.
init(ExtPrg) ->
    register(complex, self()),
    process_flag(trap_exit, true),
    Port = open_port({spawn, ExtPrg}, [{packet, 2}, binary]),
    loop(Port).
loop(Port) ->
    receive
 {call, Caller, Msg} ->
     Port ! {self(), {command, term_to_binary(Msg)}},
     receive
  {Port, {data, Data}} ->
      Caller ! {complex, binary_to_term(Data)}
     end,
     loop(Port);
 stop ->
     Port ! {self(), close},
     receive
  {Port, closed} ->
      exit(normal)
     end;
 {'EXIT', Port, Reason} ->
     exit(port_terminated)
    end.

Note that calling complex2:foo/1 and complex2:bar/1 will result in the tuple {foo,X} or {bar,Y} being sent to the complex process, which will code them as binaries and send them to the port. This means that the C program must be able to handle these two tuples.
 
C Program
 
 
The example below shows a C program communicating with an Erlang program over a plain port with home made encoding.

 /* port.c */
typedef unsigned char byte;
int main() {
  int fn, arg, res;
  byte buf[100];
  while (read_cmd(buf) > 0) {
    fn = buf[0];
    arg = buf[1];
   
    if (fn == 1) {
      res = foo(arg);
    } else if (fn == 2) {
      res = bar(arg);
    }
    buf[0] = res;
    write_cmd(buf, 1);
  }
}

Compared to the C program above used for the plain port the while-loop must be rewritten. Messages coming from the port will be on the Erlang external term format. They should be converted into an ETERM struct, a C struct similar to an Erlang term. The result of calling foo() or bar()
must be converted to the Erlang external term format before being sent back to the port. But before calling any other erl_interface function,
the memory handling must be initiated.
 
erl_init(NULL, 0);

For reading from and writing to the port the functions read_cmd() and write_cmd() from the erl_comm.c example below can still be used.
 
/* erl_comm.c */
typedef unsigned char byte;
read_cmd(byte *buf)
{
  int len;
  if (read_exact(buf, 2) != 2)
    return(-1);
  len = (buf[0] << 8) | buf[1];
  return read_exact(buf, len);
}

write_cmd(byte *buf, int len)
{
  byte li;
  li = (len >> 8) & 0xff;
  write_exact(&li, 1);
 
  li = len & 0xff;
  write_exact(&li, 1);
  return write_exact(buf, len);
}

read_exact(byte *buf, int len)
{
  int i, got=0;
  do {
    if ((i = read(0, buf+got, len-got)) <= 0)
      return(i);
    got += i;
  } while (got<len);
  return(len);
}

write_exact(byte *buf, int len)
{
  int i, wrote = 0;
  do {
    if ((i = write(1, buf+wrote, len-wrote)) <= 0)
      return (i);
    wrote += i;
  } while (wrote<len);
  return (len);
}

The function erl_decode() from erl_marshal will convert the binary into an ETERM struct.
int main() {
  ETERM *tuplep;

  while (read_cmd(buf) > 0) {
    tuplep = erl_decode(buf);

In this case tuplep now points to an ETERM struct representing a tuple with two elements; the function name (atom) and the argument (integer).
By using the function erl_element() from erl_eterm it is possible to extract these elements, which also must be declared as pointers to an ETERM

struct. fnp = erl_element(1, tuplep);
argp = erl_element(2, tuplep);

The macros ERL_ATOM_PTR and ERL_INT_VALUE from erl_eterm can be used to obtain the actual values of the atom and the integer. The atom value is represented as a string. By comparing this value with the strings "foo" and "bar" it can be decided which function to call.

    if (strncmp(ERL_ATOM_PTR(fnp), "foo", 3) == 0) {
      res = foo(ERL_INT_VALUE(argp));
    } else if (strncmp(ERL_ATOM_PTR(fnp), "bar", 3) == 0) {
      res = bar(ERL_INT_VALUE(argp));
    }

Now an ETERM struct representing the integer result can be constructed using the function erl_mk_int() from erl_eterm. It is also possible to use the function erl_format() from the module erl_format.

    intp = erl_mk_int(res);

The resulting ETERM struct is converted into the Erlang external term format using the function erl_encode() from erl_marshal and sent to Erlang using write_cmd().

    erl_encode(intp, buf);
    write_cmd(buf, erl_eterm_len(intp));

Last, the memory allocated by the ETERM creating functions must be freed.

    erl_free_compound(tuplep);
    erl_free_term(fnp);
    erl_free_term(argp);
    erl_free_term(intp);

The resulting C program is shown below:

/* ei.c */

#include "erl_interface.h"
#include "ei.h"

typedef unsigned char byte;

int main() {
  ETERM *tuplep, *intp;
  ETERM *fnp, *argp;
  int res;
  byte buf[100];
  long allocated, freed;

  erl_init(NULL, 0);

  while (read_cmd(buf) > 0) {
    tuplep = erl_decode(buf);
    fnp = erl_element(1, tuplep);
    argp = erl_element(2, tuplep);
   
    if (strncmp(ERL_ATOM_PTR(fnp), "foo", 3) == 0) {
      res = foo(ERL_INT_VALUE(argp));
    } else if (strncmp(ERL_ATOM_PTR(fnp), "bar", 17) == 0) {
      res = bar(ERL_INT_VALUE(argp));
    }

    intp = erl_mk_int(res);
    erl_encode(intp, buf);
    write_cmd(buf, erl_term_len(intp));

    erl_free_compound(tuplep);
    erl_free_term(fnp);
    erl_free_term(argp);
    erl_free_term(intp);
  }
}

 

Running the Example

 

1. Compile the C code, providing the paths to the include files erl_interface.h and ei.h, and to the libraries erl_interface and ei.

unix> gcc -o extprg -I/usr/local/otp/lib/erl_interface-3.2.1/include //
      -L/usr/local/otp/lib/erl_interface-3.2.1/lib //
      complex.c erl_comm.c ei.c -lerl_interface -lei

In R5B and later versions of OTP, the include and lib directories are situated under OTPROOT/lib/erl_interface-VSN, where OTPROOT is the root directory of the OTP installation (/usr/local/otp in the example above) and VSN is the version of the erl_interface application (3.2.1 in the example above). In R4B and earlier versions of OTP, include and lib are situated under OTPROOT/usr.

 

2. Start Erlang and compile the Erlang code.

 

unix> erl
Erlang (BEAM) emulator version 4.9.1.2

Eshell V4.9.1.2 (abort with ^G)
1> c(complex2).
{ok,complex2}

3. Run the example.

2> complex2:start("extprg").
<0.34.0>
3> complex2:foo(3).
4
4> complex2:bar(5).
10
5> complex2:bar(352).
704
6> complex2:stop().
stop