Google Panic

This post was originally mailed in to mailto:larryhsfriends@yahoogroups.com, 8 June 2008 --

Yesterday, I got a scare when I logged into GMail with Lynx. There were a bunch of check boxes, one for each message, but no link to actually view each of the messages. This put me in shock, my confidence in Google hanging in mid air just off the edge of the cliff.

I floundered around a bit to verify that I was in their basic html user interface (I was) and then looked at the actual html source code. All the links to view the messages were there, but inspection of the markup language source code showed that there were some missing closing tags with each message, the probably cause of the problem. There was no proliferation of javascript/AJAX as I had feared, it is probably just a mistake by some new CGI programmer.

I sent them some email (that still worked :-) ), explaining the problem, pointing out the exact spot and probable missing tags that belonged on the table/row of each inbox message. I think having their basic html user interface output proper html is important to Google, this is the 'Plan B' not just for text mode and other alternative browsers but the many mobile devices they want to get market penetration on.

So we'll see what happens. Dallas E. Legan II / legan@acm.org / dallas.legan@gmail.com / aw585@lafn.org http://isthereanotherquestion.blogspot.com

18 June 2008 Addenda

Since the above was blogged, I found a couple of ways around the above problem. I should of saved a sample of the HTML when the problem first occured to verify that there haven't been any changes in the mean time.
• For Lynx, ^V toggles between two different HTML parsers.
  • The apparent default parser is 'SortaSGML', which follows strict, formal standards, and is prone producing bad results when confronted by poorly written markup.
  • The alternate 'TagSoup' parser, which will put up with almost any violation of standards. This of course is the one that can deal with the new GMail format.
• After some experimentation, I found that the problem seemed to be caused by some '<div>' tags. The exact purpose of these tags seemed to somewhat unclear, especially in major browsers of the time a pocket HTML reference I use was written. A privoxy edit to eliminate them solved the problem:

  s|</?div>||igsx
  

Dallas E. Legan II / legan@acm.org / dallas.legan@gmail.com / aw585@lafn.org http://isthereanotherquestion.blogspot.com

WWWsh

• meandering back from the County library, thinking about all the Google Search commands, I realized these were just a subset of the http://Yubnub.org commands.
• This revived the idea of the World Wide Web shell, that Yubnub doesn't quit nail.
• I'd given thought to making that shell with something like Kermit or CURL.
• Some experiments with the information from the Yubnub html form indicated that the site makes extensive use of http redirection.
• Using these tools would also require using some tool to render the html when that is appropriate (most of the time), yet leave it open to output raw html when appropriate (some times)
• This points back to a web browser, with some kind of plug in, like some of the Yubnub tools listed on their web site.
• One of the main charateristics of a shell interface is that it normally returns back to the prompt, or the command entry field, when it completes running a command.
• To make sure that the you always get back to the Yubnub/WWW command prompt would require either something done at the client program or that Yubnub proxy all the final "command" output.
• One way of implimenting this idea would be to edit some chunk of html form code into all web pages, and this could be done with Privoxy.
• And so the following Privoxy filter, in /etc/privoxy/user.filter:

  
  ###################################################################### 14:24:24##
  #
  # yubnub: Try at a yubnub 'command prompt' on each web page
  #               30/12/2007 d.e.l.
  #
  #################################################################################
  FILTER: yubnub Try at a yubnub 'command prompt' on each web page
  
  
  s|(</body[^>]*>)|\
    <form action="http://www.yubnub.org/parser/parse" \n\
           + method="get" name="input_box"> \n\
      <input type="text" name="command" size="55" value=""/> \n\
    </form>  \n\
    $1|ig
  
  #  For a single text input field, Lynx  will submit on entering return,
  #    so no submit needed
  #  W3M and possibly members of the Links family browsers have
  #    function to force submission of forms
  #  To use with other browsers, you may need to include a line like
  #    this before the closing of the form with '</form>':
  #
  #   <input type="submit" value="Enter">
  #
                    
  

• then this in user.action:

   
   { +filter{yubnub} }
   .
    
  

• And after restarting Privoxy you get a screen shot like this:

  --
  ,,,                                            Los Angeles Free-Net's Home Page+
     The Los Angeles Free-Net
  
     A volunteer non-profit organization dedicated to bringing people
     together, providing community information, and offering Internet
     access at the lowest possible cost.
  
     [1]____________________________________________________
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  (Form field) Inactive. Use  to edit ('x' to submit with no cache).
  --
  

  Where the entry field is the command prompt/entry field.

• In lynx, ^E will always put at the bottom of the current web page, which from now on will be the prompt location.
• This has the obvious weakness that there must be a '</BODY..' tag for it to work, but then some apps run inside bash fail to complete properly and hang also.
• It also turned out that default settings for Privoxy skip edits on sites with 'wiki' in the address (as one example), but it is normal for a shell to not present a prompt while an application is running also, so this makes some logic.
• Time to start learning more Yubnub commands - the ones I think of off the top of my head are 'g ', 'w ' (for a weather report), 'wp ' 'ls' (for a list of commands).

Addenda 15 Jan. 2008

I should point out this idea could be used for simplified services, like Google search, standard and www.google.com/ie, google proxies like www.scroogle.com/scraper.html, etc. It's just that Yubnub includes these as commands, but if there were no Yubnub, these would certainly be fine enough to justify the idea. Also, I may need to rename this - I noticed that WWWsh is already used for some software of some kind!