Lets say that you have this kind of csv files, named like:
results_-45C_deviceA.csv
results_0C_deviceA.csv
results_30C_deviceA.csv
results_125C_deviceA.csv
results_150C_deviceA.csv
and each file contain columns with data. Each column has a unique header like this :
> cat results_30C_deviceA.csv
Vport1 Vport2 Vport3
11 123 4545
22 123 4545
33 123 4545
44 123 4545
And you want to extract all columns that contain string “Vport2” from all files but to keep the order of temperatures in the normal order ( like -45C, 0C, 30C, 125C…)
The Solution is to sort them in order and then to use PR and AWK. The PR will concatenate all files in a big file with all columns. AWK will print only the columns that have the string on the first row.
pr -m -t -s `ls | sort -n -t _ -k 2` | awk 'NR==1{for(i=1;i<=NF;i++)if($i~/Vport2/)f[n++]=i}{for(i=0;i<n;i++)printf"%s%s",i?" ":"",$f[i];print" "}'
It is only one line, but is hard to understand it .
Please notice the following:
ls = list the files from current folder
sort -n option = treat strings as numbers
sort -t _ option = consider the names as strings separated by “_” delimiter
sort -k 2 option = order the files based on 2nd field , considered as number ( -n option) , so first fill be -45C, then 0C, then 30C, then 125C, then 150C. If you do not use this option then the order will be 0, 125, 150, 30, -45
ls and sort commands are inside a “ . This is the symbol that is on the same key with ~ (top left corner )!!!!. This is not the ‘ apostrophe !
for AWK option please read the manual or search the web.
The result will be
Vport2 Vport2 Vport2 ....
123 123 123
123 123 123
123 123 123
The new file will contain no info about the name order but you know that is in good order because of “sort” command.
Other solution may be possible but this worked for me.
The AWK was inspired by this post : http://www.tek-tips.com/viewthread.cfm?qid=1497302