Bash URI parser using SED

Filed under: Command line tools, Networking

Warning! This version is now obsolete!
Check out the new and improved version (using only Bash built-ins) here!

Here is a command-line (bash) script that uses sed to split the segments of an URI into usable variables. It also validates the given URI since malformed strings produce the text “ERROR” which can be handled accordingly:

# Assembling a sample URI (including an injection attack)
uri_1='http://user:pass@www.example.com:19741/dir1/dir2/file.php'
uri_2='?param=some_value&array[0]=123&param2=\`cat /etc/passwd\`'
uri_3='#bottom-left'
uri="$uri_1$uri_2$uri_3"

# Parse URI
op=`echo "$uri" | sed -nrf "uri.sed"`

# Handle invalid URI
[[ $op == 'ERROR' ]] && { echo "Invalid URI!"; exit 1; }

# Execute assignments
eval "$op"

# ...work with URI components...

Notice the "uri.sed" file given to sed?
Read the rest of this entry »

Recursive file/directory change-detection

Filed under: Command line tools, One-liners

Version 2

Another, much faster method would be to use ls -lR to browse over the filesystem. On a newly installed Debian virtual machine (on Xen) hashing the entire filesystem (the root directory) took approximately 1.7 seconds. So, here it is:

ls -lR "$D" | sha1sum | sed 's/[ -]//g'

This method is sensitive to file name, size and modification size; usually that would be enough but if you need more control use…

Version 1

Detect when the contents of a file or directory ($D) changes:

find "$D" | while read f; do stat -t "$f"; done | sha1sum | sed 's/[ -]//g'

This yields a hash of the current state of the file or directory which is extremely sensitive to even the most subtle changes (even a simple touch to any file/directory somewhere inside "$D" changes the generated hash).

Read the rest of this entry »

Detect number of CPUs on a machine

Filed under: Command line tools, One-liners

UPDATE: Steven pointed out (very nicely) that there’s no need for cat in this picture, grep would do just fine on its own. So, thanks Steven!

Detect how many CPU cores are present on the running machine:

grep -c processor /proc/cpuinfo

This can be very useful when writing multi-threaded programs to properly match the number of threads with the number of CPU cores.

Read the rest of this entry »