-----Original Message----- From: Shawn [mailto:shawn@baimc.com] Sent: Tuesday, November 13, 2001 4:51 PM To: tommy@atrixnet.com Subject: Re: [CGI] Merging files Hey Tommy, After looking at your response, I am just wondering why a couple of simple reads and a simple write would not work... open(F,"; close(F); open(F,"; close(F); push (@file1, @file2); open(F,"newfile") or die "couldn't write newfile: $!\n"; print F @file1; close(F); Just curious, Shawn Well lets see about this. I think it's a question worth a fair bit of analysis. So how about we put up the pros against the cons and we'll decide for ourselves solution works better for us in the given circumstances.... +====================+===========================+===========================+ | Factor in | Shawn's proposed solution | Tommy's proposed solution | | Consideration | (procedural) | (object-oriented) | +====================+===========================+===========================+ | Speed of | 25 wallclock secs | 21 wallclock secs | | Execution | 20.56 usr + 2.30 sys | 5.15 usr + 2.68 sys | | | = 22.86 CPU | = 7.83 CPU | | 10,000 iterations | | | | of complete code | | | | compilation and | 25 wallclock secs | 22 wallclock secs | | execution within | 20.98 usr + 2.36 sys | 5.01 usr + 2.54 sys | | one perl system | = 23.34 CPU | = 7.55 CPU | | process. | | | | | | | | Each app had to | | | | merge 7 files, | | The numbers prove it; | | 10,000 times. The | | you can't whine that an | | execution times | | Object-Oriented style | | shown to the right | | slows down the program | | reflect the amount | | in the least. In fact, | | of time each app | | it is the very code | | took to do this | | re-use and modularity | | task. | | which account for the | | | | more speedy execution | | [note] - | | times. | | One run for either | | | | app would take | | | | somewhere in the | | | | neighborhood of | | | | a few milliseconds | | | | at most. | | | | | | | +--------------------+---------------------------+---------------------------+ | Ease of Use | Not a utility which can | May be run from the | | | be run from the command | command line. | | | line to take arguments | | | | where you specify any | Takes arguments from the | | | number of files to merge | command line to let you | | | and in what order. | customize input/output. | | | | | | | Has no built-in help, | Built in help. Runs just | | | or usage information of | like any binary on linux | | | any kind. | where an -h switch passed | | | | to the app returns a | | | Not many diagnostic error | usage guide and examples | | | messages. Not any error | of correct usage syntax. | | | messages which help find | | | | and eliminate bugs and | Accepts an unlimited num- | | | fix user mistakes such as | ber of files to merge | | | improper usage syntax | | | | with suggestions of how | Lets you specify where to | | | to remedy any errors. | write the new file, and | | | | what to name it. | | | Works from one directory | | | | only, or must have fully | Provides an abundance of | | | qualified paths to root | case-specific help in all | | | for each file it opens | its error messages and | | | | provides correct suggest- | | | | tions for the user to | | | | correct any mistakes he | | | | she had made. | | | | | | | | Includes a stack trace. | | | | | | | | Works from any directory | | | | and takes relative paths. | | | | | +--------------------+---------------------------+---------------------------+ | Stability | If ever forked or | Implements FLOCK on all | | | run in iterations from | systems that implement | | | inside a virtual environ- | the mechanism. Even | | | ment -(this is how it was | Win32 implements a form | | | tested for speed of exe- | of FLOCK. You can even | | | cution)- it will certian- | get a complete simulation | | | ly start encountering | of FLOCK-ed behavior out | | | what is called a race | of your script when run | | | condition during its | on Win32 by using Cygwin. | | | IO operations on all the | | | | files involved in the | Integrity of all files | | | script. | merged by the app can be | | | | guaranteed. | | | The integrity of all | | | | files merged by the app | | | | cannot be guaranteed | | | | | | +--------------------+---------------------------+---------------------------+ | Reusability | Use one time and throw | Use as many times as you | | | away. If you ever want | want, and never re-write | | | to use it again, you have | a single line of code-- | | | to re-write it or pay | or pay somebody else to. | | | someone else to do it. | | | | | Embeddable in other | | | The code may be small, | applications or virtual | | | but it sure seems silly | environments without | | | to spend all the time it | modification to the code. | | | would take to go back and | | | | re-write it 10,000 times | | | | before you could accomp- | | | | lish the same task as | | | | the Object Oriented | | | | solution can in one run | | | | with a list of 10,000 | | | | files. | | | | | | +--------------------+---------------------------+---------------------------+ | Scalability | Can not be re-used by | Can be re-used in other | | | other applications, can | applications and can also | | | not be automated for | be automated in large | | | quick batch jobs. | batch jobs. | | | | | +--------------------+---------------------------+---------------------------+ | Maintainability | You have to re-write the | Written once. Never has | | | code each time you need | to be re-visited to do | | | to merge new files. This | the task it was made for. | | | means there's a higher | This means that it is | | | chance for errors to | debugged one time only. | | | creep in, and | | | | necessitates frequent | | | | debugging. | | | | | | +--------------------+---------------------------+---------------------------+ | Compatibility | Will work on systems that | Works for Win32, linux | | | implement a specific type | BSD, Solaris, freeBSD, | | | of file system and could | and probably more. | | | run into problems with | | | | platforms of a different | mod_perl compatible | | | directory structure. | | | | | | | | not designed for mod_perl | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | +--------------------+---------------------------+---------------------------+ | Security | If hacked, the syntax | If exploitation is ever | | | used to open and write to | attempted, the manner | | | files would allow a | in which IO is handled | | | hacker the pleasure of | thwarts all efforts to | | | direct access to your | gain access to the system | | | system. | and never escapes to a | | | | shell under any circum- | | | Files are opened without | stance whatsoever. In | | | any kind of check for | case you were wondering, | | | what kind of file is | the procedural solution | | | about to be opened, or if | used in Shawn's example | | | it even exists. | could easily be used to | | | | let a given user escape | | | If it doesn't exist, it | commands to the shell, | | | could be that someone is | although it is not a | | | trying to get tricky on | matter of procedural vs. | | | you and open up a pipe | OO programming which | | | or a tty. | determines this vulnera- | | | | bility. It is rather a | | | We probably don't need | matter of using a safer | | | to be merging binaries | type of syntax when open- | | | either. | ing and writing to files. | | | | | +--------------------+---------------------------+---------------------------+ You decide which solution you want. Depending on your situation, it could very likely be either. To see the code which was used to test each of these solutions, take a look at the following URIs. Benchmarking app to test the procedural code: http://atrixnet.com/cgi-bin/archive.pl/pub/scripts/merge_files_procedural.txt Benchmarking app to test the Object Oriented code: http://atrixnet.com/cgi-bin/archive.pl/pub/scripts/merge_files_OO.txt Tommy Butler Internet Strategies, Inc. Everything is Possible web http://www.istrat.com email mailto:tommy@istrat.com tel 214·393·1000 ext 207 fax 800·307·8105 2200 North Lamar, Suite 307 Dallas, TX 75202