___MESSAGE HEADERS___

Return-Path: <tommy@atrixnet.com>
Received: from C1601202A (c1601202-a.alntn1.tx.home.com [24.254.174.79])
   by isi1.istrat.com (8.11.0/8.11.0) with SMTP id f8F7djU07351
   for <tommy@atrixnet.com>; Sat, 15 Sep 2001 02:39:45 -0500
Reply-To: <tommy@atrixnet.com>
From: "Tommy Butler" <tommy@atrixnet.com>
To: <tommy@atrixnet.com>
Subject: RE: Fork, Thread, or a kick to the head?
Date: Sat, 15 Sep 2001 02:49:53 -0700
Message-ID: <MABBJNNKNGCCADBAELCHAEDJCMAA.tommy@atrixnet.com>
MIME-Version: 1.0
Content-Type: text/plain;
   charset="iso-8859-1"
Content-Transfer-Encoding: 8bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
In-Reply-To: <MABBJNNKNGCCADBAELCHIEDICMAA.tommy@atrixnet.com>
Importance: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Status:

__END MESSAGE HEADERS__


I'm trying to accomplish the following with the information in perlipc and a
few articles I found at webtechniques.com but so far this isn't cutting it.
Lets do this in the free-style XML meta-language I mentioned in my previous
post.  Just for now I'll call it something like xml::CGIquestion...

<?xml::cgiquestion
 version="0.1"
 standalone="yes"?>

   <cgi_question>

      <situation>

            <whats_up>

               <brief>
               I have a main process making HTTP 1.1 GET
               queries to different servers with LWP
               </brief>

            </whats_up>

            <whats_wrong>

               <brief>
                  Execution of this procedure in a stacked or
                  looped succession is too slow.
               </brief>

               <detailed>
                  The requests are made in succession, before
                  the retrieved pages can be processed for to
                  extract the desired content.

                  The amount of time each request takes to return
                  a result is dependent on the remote machine's
                  response times.  This makes things slow, especially
                  for machines that limit the number of hits per
                  second they will accept. (a maximum of 6 queries
                  is getting made to any remote server.)

                  This is taking too long by my standards, and
                  those of my boss.
               </detailed>

            </whats_wrong>

         <fix_ideas>

            <idea
             id="1"
             type="best guess"
             tested="partially">

               <hypothesis>
                  I think it would be faster if I forked off the
                  querying process to child processes with their
                  STDOUT hooked to my STDIN and read from them
                  as they completed or gave up on those which were
                  too slow.
               </hypothesis>

               <outcome
                code="failure">
                  Documentation in perlipc and what I could
                  find in a few articles from webtechniques.com
                  is insufficient in providing me with enough
                  details about available implementation methods,
                  pros/cons of using these methods, workarounds
                  for bugs/risks/platform depencencies, documented
                  examples with source code. (now we're really
                  wishfully thinking!)
               </outcome>

            </idea>

            <idea
             id="2"
             type="last_resort"
             tested="no">

               <hypothesis>
                  I could try to make the process multi-threaded
                  by using the experimental Thread.pm but I think
                  it is clear why this is the alternative option
                  as opposed to the primary option due to the
                  critical nature of reliability that the resulting
                  code demonstrate in an enterprise mod_perl app
                  which is already several hundred thousand lines
                  of code. (now that's a JAPH!)
               </hypothesis>

               <outcome
                code="not tested">
                  Not researched very extensively thus far.  To be
                  considered as a back-up plan for now.
               </outcome>

            </idea>

         </fix_ideas>

   </situation>

      <help_request>

         <summary>
            Please provide feedback in the following forms.
         </summary>

         <details>

            <item
             id="1"
             priority="high"
             precidence="1">
               Please provide links to better known resources on the
               topic of forking, my loosely proposed course of
               action for solving this problem, or instruction first-
               hand in the process of implementing such a solution
               with source code and explanation of code.
            </item>

            <item
             id="2"
             priority="med"
             precidence="2">
               Please provide links to better known resources on the
               topic of threading, my loosely proposed alternative course
               of action for solving this problem; and if so doing, please
               offer suggestions why this would be a superior choice in
               solving the problem to other proposed methods.
            </item>

         </details>

      </help_request>

   </cgi_question>


  - Tommy Butler,
    Internet Strategies, Inc. º ° º  Everything is Possible.

    web  http://istrat.com
            email   mailto:tommy@istrat.com
                     tel   2 1 4 . 3 9 3 . 1 0 0 0  ext207
                             fax   8 0 0 . 3 0 7 . 8 1 0 5

    2200 North Lamar, Suite 307   º ° º   Dallas, TX  75202