Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

As mentioned elsewhere in this post this leads to dreadful performance. You are better off using iflag/oflag options seek_bytes, skip_bytes and count_bytes to treat these options as bytes and using a large block size.


Hm, I don't see any details on that in the man page. What's the logic there?


skip, seek and count default to counting the number of blocks. So if you want Megabytes 50-60 of a file you can do either a 1 byte block size:

> dd if=/tmp/file1 bs=1 skip=50000000 count=10000000

or you can use any block size you want and treat skip and count as byte counts, not block counts.

> dd if=/tmp/file1 bs=4M skip=50M count=10M iflag=skip_bytes,count_bytes

Option 1 will run at Kilobytes per second as it is transferring 1 byte at a time (my test gives me 933kB/s)

Option 2 will run a 100's MB per second as it is transferring 4MB at a time (my test gives me 202MB/s).

Edit: What I should have said at the top of the post was "If you want to extract data from a file whose size is not an integer block multiple". In the above example you could have used a Blocksize of 10MB for the same result. You cannot do this for an oddly size extraction (say 1234567bytes).


Interesting, thanks for the hint!

Why doesn't the man page say anything about these flags? They are completely obscure becasue of that.


I only found out because I use dd 1000's of times each day and have had to prototype some really esoteric data pipelines using it. I have even compiled a few custom versions to do various tasks.

One other nuggets of wisdom which is very poorly documented in the man page: if you have a long running dd process (I often have 12+ hr dd processes reading 10+TB tapes) you can send a USR1 signal to check the progress. Very useful to check if your tape drive has had a hardware failure.

Information on both this and the byte counts are in my man page (GNU coreutils 8.30 on Linux Mint 20) around lines 130.


I knew about USR1 and it's in the man for me too, but I don't see anything about skip_bytes / count_bytes flags, which is strange.

Debian testing, coreutils 9.1.


I am running Coreutils 8.3 on Linux mint 20.

It is also in Coreutils 8.22 in RHEL7.

https://man7.org/linux/man-pages/man1/dd.1.html

Edit: I just found this in the coreutils 9.0 changelog:

  dd now counts bytes instead of blocks if a block count ends in "B".
  For example, 'dd count=100KiB' now copies 100 KiB of data, not
  102,400 blocks of data.  The flags count_bytes, skip_bytes and
  seek_bytes are therefore obsolescent and are no longer documented,
  though they still work.


Thanks for the pointer.

So you can simply do something like:

   dd if=foo skip=nB count=mB bs=4M of=bar
To get m bytes from offset n




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: