my ideas in action

Take all columns that have a specific header

Lets say that you have this kind of csv files, named like:


and each file contain columns with data. Each column has a unique header like this :

> cat results_30C_deviceA.csv
Vport1 Vport2 Vport3
11 123 4545
22 123 4545
33 123 4545
44 123 4545

And you want to extract all columns that contain string “Vport2” from all files but to keep the order of temperatures in the normal order ( like -45C, 0C, 30C, 125C…)

The Solution is to sort them in order and then to use PR and AWK. The PR will concatenate all files in a big file with all columns. AWK will print only the columns that have the string on the first row.

 pr -m -t -s `ls | sort -n -t _ -k 2` | awk 'NR==1{for(i=1;i<=NF;i++)if($i~/Vport2/)f[n++]=i}{for(i=0;i<n;i++)printf"%s%s",i?" ":"",$f[i];print" "}'

It is only one line, but is hard to understand it .

Please notice the following:

ls = list the files from current folder

sort -n option = treat strings as numbers

sort -t _ option = consider the names as strings separated by “_” delimiter

sort -k 2 option = order the files based on 2nd field , considered as number ( -n option) , so first fill be -45C, then 0C, then 30C, then 125C, then 150C. If you do not use this option then  the order will be 0, 125, 150, 30, -45

ls and sort commands are inside a “ . This is the symbol that is on the same key with ~  (top left corner )!!!!. This is not the ‘ apostrophe  !

for AWK option please read the manual or search the web.

The result will be

Vport2 Vport2 Vport2 ....
123 123 123
123 123 123
123 123 123

The new file will contain no info about the name order but you know that is in good order because of “sort” command.

Other solution may be possible but this worked for me.

The AWK was inspired by this post :


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: