Though I think 720p is towards the low end of resolutions this'll work on -- the cimbar code itself is 1024x1024, and while there's some wiggle room (hamming distance between the symbols), in my experience it gets pretty iffy below 900x900.
Also... I should probably update the overlay to auto-hide when a file is selected -- it helps a lot to have the full color range.
You've seen QR codes, maybe Microsoft HCCB [0], maybe jabcode [1] – well, here's a prototype of something new for the pile. :)
I saw txqr [2] a while back and was impressed, but also curious about how much throughput was possible with animated bar codes. I may have gotten carried away in my research into the question.
cimbar is a single and multi-frame color barcode format using reed solomon (for now) and wirehair [3] for error correction. Files are encoded into an animated series of bar codes drawn to the display. Files are decoded by an Android app [4] with its camera pointed at the cimbar code. This works with all antennas off, e.g. in airplane mode, because it's only using the visual data channel.
Sustained transfer speed is currently on the order of 800 kilobits/second. So it's not very practical for files larger than a couple of MB, unless you have a lot of free time. :)
Amazing! Do you know any variants of these which squeeze out the maximum transfer rate over a high quality connection? Thinking about data transfer from a VPN/RDP remote desktop where I want to grab the screen output on the client with very little loss in quality (basically only limited by the chosen video encoder).
I don't think air-gapped audio transmission could ever reach a fast and reliable transmission so that it allows transferring files in reasonable amount of time over reasonable transmitter-receiver distances. It's just too many hardware and physical limitations for this approach.
Having said that, I am actually working on a small library for data-over-sound which can be used for small data chunk transmissions across the room [0].
Well, there's diminishing returns in format shifting. The encoded barcode contains various types of quasi-redundant visual information (e.g. error correction codes) to allow decoding to happen, so for audio-based transfer it'd be better to skip the image encode and blast out the file directly.
That said... given the somewhat remarkable way fountain codes work, there's nothing stopping us from having a protocol that uses the audio and the video channels simultaneously for better throughput...
Would it be beneficial to use one channel (either audio or visual) to transmit the information and the other one for responses like acknowledge? So kind of like TCP over two different channels?
Well, as for acks specifically -- cimbar itself doesn't really need them, thanks to fountain codes [0].
But I can imagine a reverse (request?) channel being useful, if it had enough bandwidth for the desired application. :)
As /u/ggerganov notes elsewhere in this thread (with some expertise on the audio side -- I can't claim any), the bandwidth of any audio channel is probably going to be pretty bad.
edit: Notwithstanding how viable of an idea it might be, HTTP over audio+video would be pretty neat. :)
Definitely. I had to draw a line somewhere for an initial "does anyone care about this?" release, so there's some rough edges here.
There are a number of loose ends that I'd like to look into down the road, and more flexibility for different use cases is on the list.
edit: I should mention: the main reason the decoder is a native Android app and not a WebAssembly app is that decoding performance is a throughput bottleneck. I wasn't too eager to pay the wasm tax, when I wasn't sure if even native performance would be good enough. As it happens, I now think decode performance is going to be ok -- but it's still a bit of a weak spot in the scheme.
I just dragged a file into the site on my desktop computer and it worked. OP said the site uses the wasm implementation so any browser with javascript enabled should be able to encode files.
Very interesting but couldn't get it to work with my smartphone. Even with a 250Ko image file, I got bored before it finished !
My camera may be too low quality and/or my monitor too small.
A barcode size parameter may be useful (even though it reduces throughput) so that it is easier to scan on all devices.
max_ | 5 years ago
I heard ordinary QR codes can hold up to 3KB of data.
How much data can this one hold?
ggerganov | 5 years ago
I tried the browser encoder and here is what it looks like:
https://youtu.be/RkpVhW7SuRY
Wonder if someone can decode the .mp4 file from the video. It's not rickroll, I promise ;-)
[OP] sz3 | 5 years ago
I like the way you think. :)
Though I think 720p is towards the low end of resolutions this'll work on -- the cimbar code itself is 1024x1024, and while there's some wiggle room (hamming distance between the symbols), in my experience it gets pretty iffy below 900x900.
Also... I should probably update the overlay to auto-hide when a file is selected -- it helps a lot to have the full color range.
[OP] sz3 | 5 years ago
Hello all,
You've seen QR codes, maybe Microsoft HCCB [0], maybe jabcode [1] – well, here's a prototype of something new for the pile. :)
I saw txqr [2] a while back and was impressed, but also curious about how much throughput was possible with animated bar codes. I may have gotten carried away in my research into the question.
cimbar is a single and multi-frame color barcode format using reed solomon (for now) and wirehair [3] for error correction. Files are encoded into an animated series of bar codes drawn to the display. Files are decoded by an Android app [4] with its camera pointed at the cimbar code. This works with all antennas off, e.g. in airplane mode, because it's only using the visual data channel.
I ported the encoder to wasm, because I could: https://cimbar.org
Sustained transfer speed is currently on the order of 800 kilobits/second. So it's not very practical for files larger than a couple of MB, unless you have a lot of free time. :)
[0] https://en.wikipedia.org/wiki/High_Capacity_Color_Barcode
[1] https://github.com/jabcode/jabcode
[2] https://github.com/divan/txqr
[3] https://github.com/catid/wirehair
[4] https://github.com/sz3/cfc/releases/latest
ktpsns | 5 years ago
Amazing! Do you know any variants of these which squeeze out the maximum transfer rate over a high quality connection? Thinking about data transfer from a VPN/RDP remote desktop where I want to grab the screen output on the client with very little loss in quality (basically only limited by the chosen video encoder).
mleonhard | 5 years ago
Using it to transfer files over a remote desktop connection is like a graphical version of zmodem [0].
[0] https://en.wikipedia.org/wiki/ZMODEM
ignoramous | 5 years ago
As a possible tangent, convert those images to inaudible audio to transfer files over the air: https://github.com/ganny26/awesome-audioqr
(air-gapped computers be damned)
ggerganov | 5 years ago
I don't think air-gapped audio transmission could ever reach a fast and reliable transmission so that it allows transferring files in reasonable amount of time over reasonable transmitter-receiver distances. It's just too many hardware and physical limitations for this approach.
Having said that, I am actually working on a small library for data-over-sound which can be used for small data chunk transmissions across the room [0].
[0] https://github.com/ggerganov/ggwave
[OP] sz3 | 5 years ago
Well, there's diminishing returns in format shifting. The encoded barcode contains various types of quasi-redundant visual information (e.g. error correction codes) to allow decoding to happen, so for audio-based transfer it'd be better to skip the image encode and blast out the file directly.
That said... given the somewhat remarkable way fountain codes work, there's nothing stopping us from having a protocol that uses the audio and the video channels simultaneously for better throughput...
rmetzler | 5 years ago
Would it be beneficial to use one channel (either audio or visual) to transmit the information and the other one for responses like acknowledge? So kind of like TCP over two different channels?
[OP] sz3 | 5 years ago
Well, as for acks specifically -- cimbar itself doesn't really need them, thanks to fountain codes [0].
But I can imagine a reverse (request?) channel being useful, if it had enough bandwidth for the desired application. :)
As /u/ggerganov notes elsewhere in this thread (with some expertise on the audio side -- I can't claim any), the bandwidth of any audio channel is probably going to be pretty bad.
edit: Notwithstanding how viable of an idea it might be, HTTP over audio+video would be pretty neat. :)
[0] https://en.wikipedia.org/wiki/Fountain_code
anonytrary | 5 years ago
You might want to have a picture on the main README as in https://github.com/sz3/libcimbar/blob/master/DETAILS.md. Had to fish for a picture...
[OP] sz3 | 5 years ago
Probably a good idea. Noted.
mleonhard | 5 years ago
Is it really phone CPU performance that limits the bandwidth? I expected it would be the camera.
VikingCoder | 5 years ago
I'm a HUGE fan of air-gap data transfer, and this is very nearly what I want.
Curious though, can't you make a web app to record the video, too?
[OP] sz3 | 5 years ago
Definitely. I had to draw a line somewhere for an initial "does anyone care about this?" release, so there's some rough edges here.
There are a number of loose ends that I'd like to look into down the road, and more flexibility for different use cases is on the list.
edit: I should mention: the main reason the decoder is a native Android app and not a WebAssembly app is that decoding performance is a throughput bottleneck. I wasn't too eager to pay the wasm tax, when I wasn't sure if even native performance would be good enough. As it happens, I now think decode performance is going to be ok -- but it's still a bit of a weak spot in the scheme.
donclark | 5 years ago
Is there a video of this in action?
learn_more | 5 years ago
It's simple to try out by uploading a file via the web page at https://cimbar.org
6510 | 5 years ago
I have no android device.
desdiv | 5 years ago
I just dragged a file into the site on my desktop computer and it worked. OP said the site uses the wasm implementation so any browser with javascript enabled should be able to encode files.
Decoding needs the Android app.
[OP] sz3 | 5 years ago
Hmm... no, but that's a good idea!
cors-fls | 5 years ago
Very interesting but couldn't get it to work with my smartphone. Even with a 250Ko image file, I got bored before it finished ! My camera may be too low quality and/or my monitor too small.
A barcode size parameter may be useful (even though it reduces throughput) so that it is easier to scan on all devices.