Problem solve Get help with specific problems with your technologies, process and projects.

Using AWK to convert a text file in Unix

My Unix box will receive a comma-delimited text file via FTP on a periodic basis. I will need to process (convert)...

this text file into another text file for further processing by an app.

For this text file conversion I will need to:
a) extract certain fields from source file
b) embed a text string at front of each rec
The above will require some IF-THEN-ELSE logic.

The new text file will have space(s) between fields.

What is the "best" common Unix utility to use for this task; e.g., shell scripting, Perl, Python, AWK, a 3-GL language (e.g., FORTRAN, C/C++), etc. (I have been wanting to get into Python some). Is Python generally standard with Unix installs or require a separate purchase or download?

Any other options you can think of with assumption that, for example, a relation database is not available?

Thank You!
I like AWK the best for this kind of thing. You can execute simple type AWK programs right at the Unix prompt. Here is an example of something that might work for you:

awk '/string/ {"yourtextstring"  print $2 }' < test_input_file > output_file

This would first search for all records in the file that have the /string/. Then it would would extract all lines with a character sequence matching that pattern, parse each line into fields separated by blanks, put "yourteststring" in the first field, and output the second field in the file. Play around with this a bit. The general Unix awk man page is a good start. Here's a nice little link worth looking at: http://www.engr.utk.edu/ecc/unix/man/awk.php

If you want to set-up flow control using if-then-else logic, here is your format at a high level.

if (condition) 
  { commands1 } 
 [ else 
  { commands2 } ] 

Though there is much you can do at the beginner's level, you will learn awk best by trying many different things with examples. If you want a good book to help you, not only with awk, but sed, another strong text manipulation tool, check out this link, from O'Reilly. They have really good Unix books, and this is one of many: http://www.oreilly.com/catalog/sed2/

Good luck!

This was last published in July 2003

Dig Deeper on Linux servers



Find more PRO+ content and other member only offers, here.

Have a question for an expert?

Please add a title for your question

Get answers from a TechTarget expert on whatever's puzzling you.

You will be able to add details on the next page.

Start the conversation

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.