COBS decoding support

Might be possible to add the encoding and decoding.

However I am not seeing how this would be actually used in a terminal with respect to data entry of the packets and how the packets would be displayed on the screen.

How do you visualise it working?
What sort of data is in the packets and how would it be entered?
How long are these packets?

Is anyone else interested?

Here's some documentation I wrote up on COBS back when the I found the wikipedia page to be confusing. I've included some information on optional enhancements.

Sources
http://conferences.sigcomm.org/sigcomm/1997/papers/p062.pdf
https://en.wikipedia.org/wiki/Consistent_Overhead_Byte_Stuffing

Note that the current [Dec 2015] version of the wikipedia article provides a confusing explanation. For now, the Cheshire / Baker article is probably clearer.

Problem
8/10-bit serial data protocols sending 8-bit binary data create synchronization problems (P6 serial, inter-board communications, etc.) Common solutions include BREAK / IDLE detection or framing (header / checksum) patterns. BREAK and IDLE detection require low-level UART access - high-level (Windows) APIs focus on minimizing data processing overhead, so control & timing "details" are often unavailable. Framing sequence implementations often need to backtrack and resynchronize, since they are subject to false positives caused by the framing pattern appearing in the data.

Solution
COBS simplifies framing by encoding data so that the framing (packet delimeter) 00-byte value is unused. It does not address issues of data integrity, which could be achieved by adding a checksum prior to encoding.

The basis of COBS encoding is that any sequence of non-zero data, followed by a 00-byte, is modified by prefixing a length byte whose value is 1 more than the number of non-zero bytes, and dropping the trailing 00-byte.

The encoding of several such (hex byte) sequences is shown below:

11 22 33 00 ... => 04 11 22 33 ...
00 11 22 33 00 44 00 ... => 01 04 11 22 33 02 44 ...
11 22 33 00 00 ... => 04 11 22 33 01 ...

Of course, not all packets end with a 00-byte - especially after a checksum is appended - so COBS unconditionally adds a 00-byte prior to encoding, and unconditionally removes it when decoding. This added byte gives the protocol its name. The encoded example packets below include both the encoded "overhead" 00-byte and the delimeter (framing) 00-byte:

00 => 01 01 00
11 22 00 33 => 03 11 22 02 33 00

Long payloads may contain runs of non-zero data which exceed the capacity of the length byte. To address this, an FF length indicates 254 non-zero bytes without a trailing 00-byte.

Implementation
COBS is easily implemented in software, especially when compared to bit-stuffing algorithms like HDLC, which are more suited to hardware implemenatations.

Although a COBS decoder must be well-behaved when fed corrupt data (i.e. length bytes whose value exceeds remaining data bytes), it can probably assume that any 00-bytes will have already been interpreted as delimeters by the receiver.

size_t cobs_decode(const unsigned char * input, size_t length, unsigned char * output)
{
size_t read_index = 0;
size_t write_index = 0;
unsigned char code;

while (read_index < length)
{
    // length of non-zero + trailing 0
    code = input[read_index];


    // make sure there's room (don't error on final 0 pad)
    if (read_index + code > length && code != 1)

        return 0;


    read_index++;


    // copy non-zero data
    for (unsigned i = 1; i < code; i++)
        output[write_index++] = input[read_index++];


    // add trailing 0 (except final 0 pad or non-zero seq > max code)
    if (code != 0xFF && read_index < length)
        output[write_index++] = 0;
}


return write_index;

}

Optional Features
If a non-zero delimeter byte is desired, each COBS-encoded output byte can be XORed with the value of the desired delimeter byte value prior to transmission. The receiver would repeat the (symmetrical) XOR operation before passing the data to the COBS decoder.

A further optimization of the COBS algorithm is zero-pair elimination. The modified algorithm works as described above, except that an E0 length indicates the (max) 223 non-zero bytes without a trailing 00-byte, and E1 thru FF lengths indicate a sequence of 0 - 30 non-zero bytes followed by two 00-bytes.

Another optimization, called COBS/R (Reduced), replaces the last length byte with the last data byte, as long as the value of the data byte is larger than that of the length byte. The decoder is able to detect this situation since the value of the expected length byte exceeds the number of remaining (non-zero) data bytes.

I assume you would want this to be a DisplaysAs choice which displays Hex bytes as the format?
Or do you you see it as a Binary Sync option that is separate from the DisplayAs choice?

Would it need to have the optional features to be useful?

BTW, adding the encoder/s as a CRC is probably easy. Post the encoder sample, and I may add it when next CRC work is done

on the send side it will be a CRC option, that fits the structure.

Can you provide encoder code?

on the receive side, realterm currently choose between raw (hex) data or interpreted as ASCII. COBS would add another option, where the hex stream would be shown as 0-delimited packets, 1 per line
on the transmit side, there could be COBS-encoding of individual packets specified in the Send tab, as well as from a file where each packet is delimited by a CR/LF newline