Writing binary files with bash

#!/bin/bash

num2bin() {
    out=$(printf %.$(($2*2))x\\n $1 | sed 's/\([0-9a-f][0-9a-f]\)/\\x\1/g')
    echo $out
    #echo $out | awk -F '' '{ printf $4$3$2$1 }'
}
num2bin 40 4 | dd of=test.txt bs=1 count=4 seek=0 conv=notrunc

 od -An -j0 -vtx4 -N4  ./test1.txt

od -An -j0 -vtx4   ./test1.txt

#!/bin/bash

num2bin() {
        printf $(printf %.$(($2*2))x\\n $1|
                        sed 's/\([0-9a-f][0-9a-f]\)/\\x\1/g')|
                awk -F '' '{ printf $4$3$2$1 }'
}

num2bin 4242 4 | dd of=test1.txt bs=1 count=4 seek=0 conv=notrunc
num2bin 4243 4 | dd of=test1.txt bs=1 count=4 seek=4 conv=notrunc

Writing binary files with bash

Hello,
I'm trying to see if I'm able to write some binary file using bash. So, when writing a binary file you often want to write a number into binary format. I ended up with this simple function for writing numbers in little endian:

function num2bin() {
   printf $(printf %.$(($2*2))x\\n $1|
        sed 's/\([0-9a-f][0-9a-f]\)/\\x\1/g')|
        awk -F '' '{ printf $4$3$2$1 }'
}

The first parameter is the number to write, the second is the number of bytes (up to 4). For example "num2bin 40 4" will output a 4-byte long string containing the number 40 in little endian.

How do we use it? I wrote an example script for creating a wav file with noise (according to wav specifications) that you can read here.

Let me know if you have a simpler version of the num2bin function.

I'm trying to find two bytes inside binary file, then increase value of those two bytes and replace them inside file. Those two bytes are on positions 0x82-0x83. For now on I have successfully extracted those two bytes using this:

#!/usr/bin/env bash
BYTES=$(tail -c +131 "$1" | head -c 2)

Those bytes have value: 1B 1F. I'm stuck with:

    How to convert bytes to integer? It should be 6943 decimal.
    How to append / echo binary data to file
    How to write increased bytes inside file on positions 0x82-0x83. I could use head -c 130 original.bin >> new_file.bin && magic_command_writing_bytes_to_file >> new_file.bin && tail -c +133 original.bin, but there must be better way.

I could do that in PHP, it should be easier, but I'm interested how to do this in bash.
text-processing conversion replace binary
shareimprove this question
    
edited Aug 30 '13 at 13:25
slm
63.4k845115
    
asked Aug 30 '13 at 11:03
piotrekkr
1756
    add comment
2 Answers
active oldest votes
up vote 4 down vote accepted
    

Testing with this file:

$ echo hello world > test.txt
$ echo -n $'\x1b\x1f' >> test.txt
$ echo whatever >> test.txt
$ hexdump -C test.txt
00000000  68 65 6c 6c 6f 20 77 6f  72 6c 64 0a 1b 1f 77 68  |hello world...wh|
00000010  61 74 65 76 65 72 0a                              |atever.|
$ grep -a -b --only-matching $'\x1b\x1f' test.txt
12:

So in this case the 1B 1F is at position 12.

    Convert to integer (there is probably an easier way)

    $ echo 'ibase=16; '`xxd -u -ps -l 2 -s 12 test.txt`  | bc
    6943

    And the reverse:

    $ printf '%04X' 6943 | xxd -r -ps | hexdump -C
    00000000  1b 1f                                             |..|
    $ printf '%04X' 4242 | xxd -r -ps | hexdump -C
    00000000  10 92                                             |..|

    And putting it back in the file:

    $ printf '%04X' 4242 | xxd -r -ps | dd of=test.txt bs=1 count=2 seek=12 conv=notrunc
    2+0 records in
    2+0 records out
    2 bytes (2 B) copied, 5.0241e-05 s, 39.8 kB/s

    Result:

    $ hexdump -C test.txt
    00000000  68 65 6c 6c 6f 20 77 6f  72 6c 64 0a 10 92 77 68  |hello world...wh|
    00000010  61 74 65 76 65 72 0a                              |atever.|

=========================================================

Binary to hexadecimal and decimal in a shell script

up vote 7 down vote favorite 2	I have a context where I need to convert binary to hexadecimal and decimal and viceversa in a shell script. Can someone suggest me a tool for this? shell-script binary arithmetic share improve this question editedFeb 19 '13 at 9:53 Gilles 182k 23 232 485 asked Feb 19 '13 at 6:40 Bangi 38 1 3
	2   Seeing the recent answers, maybe you should specify whether “binary” meansbinary number or binary file.–  manatwork Feb 19 '13 at 7:08 add comment

5 Answers

active oldest votes

up vote 8 down vote accepted

It's fairly straightforward to do the conversion from binary in pure bash (printf is a builtin): Binary to decimal $ echo "$((2#101010101))" 341 Binary to hexadecimal $ printf '%x\n' "$((2#101010101))" 155 Going back to binary using bash alone is somewhat more complex, so I suggest you see the other answers for solutions to that. share improve this answer editedSep 29 '13 at 20:42 answered Feb 19 '13 at 7:40 Chris Down 30.6k 2 57 102

how about decimal to hexadecimal.–  Bangi Feb 19 '13 at 8:54 @Bangi Just doprintf '%x\n' 10, for example.–  Chris Down Feb 19 '13 at 10:14 What's the point of usingprintf %d for bin2dec?$(( ... )) already gives a decimal result,echo is enough.–  Andrea Corbellini Feb 19 '13 at 13:42 @AndreaCorbellini - Personal preference. In general, I useprintf '%s\n' foo instead ofecho for a variety of reasons (mostly portability), for the same reason, I don't use it here.–  Chris Down Feb 20 '13 at 5:24 @ChrisDown: I though this question was explicitly about Bash (which has a well-implementedecho builtin). My bad!–  Andrea Corbellini Feb 20 '13 at 9:49 add comment

up vote 3 down vote

Assuming that by binary, you mean binary data as in data with any possible byte value including 0, and not base-2 numbers: To convert from binary, od (standard), xxd (comes withvim) orperl'sunpack come to mind. od -An -vtu1 # for decimal od -An -vtx1 # for hexadecimal xxd -p # for hexa perl -pe 'BEGIN{$\="\n";$/=\30};$_=unpack("H*",$_)' # like xxd -p # for decimal: perl -ne 'BEGIN{$\="\n";$/=\30;$,=" "}; print unpack("C*",$_)' Now, to convert back to binary, awk (standard), xxd -r or perl's pack: From the decimal output from od -tu1 or perl above: awk '{for (i = 1; i <= NF; i++) printf "%c", $i}' perl -ape '$_=pack("C*",@F)' From the hexa perl or xxd -p above: xxd -r -p perl -pe 'chomp;$_=pack("H*",$_)'