Advanced Web Development by AskApache is a Firefox Collection I created for myself to make finding and installing my favorite Firefox Add-ons simple and easy.

My Setup

As this screenshot shows, I only use a handful of add-ons at a time. These buttons allow me to clear the DNS/Cookies/Cache for whichever site I'm on when I click it. Very very helpful for me as a web designer. The 4th button there is just a restart button. Other than those, Firebug, YSlow, LastPass, and Web Developer are the only ones I always use regularly.

I like the idea of the last.fm but it's not as powerful as the site, which is awesome. Lately I've been listening to Kings of Leon Radio...

Slow Firefox

The more add-ons you have, disabled or not, the slower Firefox is. (unless you are running your profile folder in TMPFS). Also, bookmarks and settings like that have a performance hit. I have been using Firefox since it launched way back when, and I have always kept my bookmarks when moving to new machines and new installations.. So with over 5 thousand bookmarks I finally did some debugging and discovered that was a huge cause of Firefox acting slow. Now I am trying to migrate them all over to Google's Gmarks, which knowing Google will be awesome eventually.

Multiple Firefox Installations, Sorta

The solution to all these problems is to use separate Firefox profiles which use separate folders to store your profile-specific extensions and settings. So I have profiles with upwards of 40 Addons installed and enabled, and another profile that is built for speed... It's very very slow to be running Firebug and have multiple tabs open.. You can use the profilemanager to load them specific profiles from the command line. I personally use separate icons on my Windows Quick Launch that I just modified the shortcut pointing to firefox to also have the profile commandline. Note also that you can have multiple profiles open and running simultaneously.. This lets you do some crazy networking and other random hacks like having many Firefox instances running each of which is configured to use a separate SOCKS Proxy or network interface, so you can really open up those pipes for some intense txrx.

You can do some very powerful things with Firefox that most people are unaware of, if you are interested start with these:

• Setting up an extension development environment
• Firefox Command Line Options

About This Collection

Web Development Add-ons for Advanced Web Developers. I personally use these to work on Sites, Servers, WordPress, Javascript, PHP, CSS, XHTML, validation, page-loading, SEO, optimizing, and much more. Add this collection.

Created by:AskApache

Updated:September 22, 2009

Share this Collection

• Digg this!
• Post to Facebook
• Add to Delicious
• Post to MySpace
• Share on FriendFeed
• Post to Twitter

Firefox Add-ons for Web Developers originally appeared on AskApache.com

Sometimes there is an urgent need for creating an exact duplicate or "mirror" of a web site on a separate server. This could be needed for creating Round Robin Setups, Load-Balancing, Failovers, or for just plain vanilla backups. In the past I have used a lot of different methods to copy data from one server to another, including creating an archive of the whole directory and then using scp to send the file over, creating an archive and then encrypting it and then sending that file over using ftp, curl, etc., and my persistence at learning new ways to do things has paid off because now I use rsync to keep an exact replica of the entire directory on an external server, without having to use all the CPU and resources of other mirroring methods.

For this article I will show how I setup a web mirror of static.askapache.com from my DreamHost server to my HostGator Server. For the transfer and synchronization of the directories we will be using rsync over SSH. We will also be automating this task using a cronjob with no user-interaction, so creating public and private keys for the ssh will be neccessary. Finally, I provide a simple shell script that prevents anyone from logging into your account with the created keys for any purpose other than to rsync.

Rsync Synchronization Magic

rsync

rsync is an open source file transfer program for Unix systems that uses the "rsync algorithm" which provides a very fast method for synchronizing files and directories from one location to another while minimizing data transfer using delta encoding when appropriate. An important feature of rsync not found in most similar programs/protocols is that the mirroring takes place with only one transmission in each direction.. It does this by sending just the differences in the files across the link, without requiring that both sets of files are present at one of the ends of the link beforehand.

Securing Rsync with SSH

I NEVER transfer any unencrypted data around unless that data is transported encrypted and to a trusted recipient, (I use HTTPS for WordPress administration) and I haven't had time to probe the HostGator system for security issues yet, so right away I decided I needed an automated way to securely transfer static.askapache.com files TO hostgator, while not allowing my hostgator account access back on my main server. So if the hostgator account were to get hacked somehow, the cracker/spammer wouldn't have access back to my main server.

Generate Keys with No Password

First I created a private key, specifically a passwordless host key, meaning to gain access with ssh you only need to supply the key, not a password+key like normal.

[local@dreamhost] $ mkdir -p ~/.ssh && chmod 700 ~/.ssh
 
# Create the public and private keys
# public key at: z.askapache-hostgator.id_rsa.pub
# private key at: z.askapache-hostgator.id_rsa
[local@dreamhost] $ ssh-keygen -t rsa -b 2048 -f ~/.ssh/z.askapache-hostgator.id_rsa
 
# add the public key to remote hosts ~/.ssh/authorized_keys file
[local@dreamhost] $ ssh-copy-id -i ~/.ssh/z.askapache-hostgator.id_rsa.pub remoteuser@remotehost
 
# or use scp + ssh to add the public key
[local@dreamhost] $ scp ~/.ssh/z.askapache-hostgator.id_rsa.pub remoteuser@remotehost:/home/remoteuser/
[local@dreamhost] $ ssh remoteuser@remotehost
[gatoraskapache@gator] $ mkdir -p ~/.ssh && chmod 700 ~/.ssh
[gatoraskapache@gator] $ cat ~/z.askapache-hostgator.id_rsa.pub >> ~/.ssh/authorized_keys
[gatoraskapache@gator] $ chmod 600 ~/.ssh/authorized_keys

Custom SSH Connection Info

This helps alot, by adding this to the very top of my ~/.ssh/config file I don't have to add all this stuff to the rsync command-line. Basically when I reference connecting to the host 'gator' it uses all these options. Very helpful and you can add as many entries as you want.

Host gator
   IdentityFile ~/.ssh/z.askapache-hostgator.id_rsa
   Port 2222
   Protocol 2
   User gatoraskapache
   HostName gator555.hostgator.com
   PasswordAuthentication no

Creating Cronjob for Synchronization

This cronjob runs every 30 minutes, copying all modified files from my local directory ~/static.askapache.com/ to the remote directory ~/public_html/z/. First I backup the current crontab, then I edit the crontab and add this.

crontab -l > .crontab-`date +%F.backup`; crontab -e

*/30 * * * * /usr/bin/rsync -e 'ssh' -rt --delete ~/static.askapache.com/ gator:'~/public_html/z/' 1>/dev/null
@midnight /usr/bin/find ~/static.askapache.com/ -type d ! -perm 755 -exec chmod 755 {} \; 1>/dev/null
@midnight /usr/bin/find ~/static.askapache.com/ -type f ! -perm 644 -exec chmod 644 {} \; 1>/dev/null

Those 2 find commands scheduled to run at midnight simply fix and permissions on files and directories in my static folder. They are all static files so there is no reason they need to have any other permission.

Only Allow rsync

This is SWEET! If you like candy that is.. It's called each time anything logs into your machine using the password less key we created above, and it simple checks what command the login process is attempting to issue. To set this up you need to edit the ~/.ssh/authorized_keys file on the remote host and prefix your public key that you added with a command directive to execute a script:

command="/home/remoteuser/validate-rsync.sh" ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwBhj6UCS7JbJ08C8pWJqCh2iXZMN7tXpYZh47f4gZZBwrNHZQ== localuser@dreamhost

validate-rsync.sh

For rsync requests it will always be rsync --server at the start of the command, so if the command is anything else then this script:

1. Sends you an email notifying you somethings up..
2. Moves the ~/.ssh folder to ~/.locked-ssh
3. Adds cronjob to move the folder back in about an hour.

#!/bin/bash
# Author: http://www.askapache.com
# Version: 1.2
# Date: 04-08-2009
 
# If the command used to login to ssh correctly starts with 'rsync --server'
# then exit this script and dont process the rest of the script
case "$SSH_ORIGINAL_COMMAND" in 'rsync --server'*) exit 0; ;; esac;
 
# the home directory where the .ssh folder is located
H=/home/remoteuser
 
# if there is a locked ssh folder, kill the rsync and die
[[ -d $H/.lssh ]] && echo "REJECTED" && exit 1
 
EMAIL=webmaster@askapache.com # notified about locked status
OC=$H/old-crontab.txt # the original crontab
NC=$H/new-crontab.txt #the new crontab
 
# When to unlock the rsync
UNLOCK_AT=$(( date +%M\ %k --date='30 minute 1 hour' ))
 
# move the .ssh to .lssh which locks all key-logins
mv $H/.ssh $H/.lssh
 
# mail a notice to the boss
echo $SSH_CONNECTION | mail -s 'RSYNC LOCKED!!!' "$EMAIL"
 
# subshell backs-up crontab then deletes active cron
(
 
 crontab -l > $OC &>/dev/null || echo -n #backup current crontab
 crontab -r >/dev/null 2>&1 || echo -n # delete current crontab
)
 
# subshell creates new crontab combined with old crontab
(
 # create new crontab
 echo -en "MAILTO='${EMAIL}'\n${UNLOCK_AT} * * * mv $H/.lssh $H/.ssh" >> $NC
 echo -n " && date|mail -s 'UNLOCKED!!!' '${EMAIL}' && crontab $OC || rm $OC && rm $OC" >> $NC
 
 # add old crontab to new crontab minus any MAILTO lines
 cat $OC | sed '/^MAILTO/d' >> $NC
 
 # load the new crontab and if it doesnt work notify boss
 crontab $NC || echo "manually mv .lssh to .ssh" mail -s 'CRONTAB PROBLEM!!!' "$EMAIL"
 
 # remove new crontab
 rm $NC
)
 
exit $?

Here is the cronjob entry it creates... This will enable the rsync again by moving the folder back, then it mails you to alert you that its back up, and finally the original crontab is restored.

MAILTO="askapache@gmail.com"
03 9 * * * mv ~/.locked-ssh ~/.ssh && date|mail -s 'RUNLOCKED!!!' "webmaster@askapache.com" && crontab ~/old-crontab.txt

Rsync/SSH Debugging and Stats

Adding the option -v to the ssh command, ie rsync -e 'ssh -vv' will give you a lot of debugging info.

By adding --stats to your rsync command you can get a detailed look at its efficacy.

Number of files: 14900
Number of files transferred: 0
Total file size: 1456832331 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 320551
File list generation time: 17.393 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 320571
Total bytes received: 20
 
sent 320571 bytes  received 20 bytes  17329.24 bytes/sec
total size is 1456832331  speedup is 4544.21

Rsync Algorithm

The rsync utility uses an algorithm (invented by the Australian computer programmer Andrew Tridgell) for efficiently transmitting a structure (such as a file) across a communications link when the receiving computer already has a different version of the same structure.

The recipient splits its copy of the file into fixed-size non-overlapping chunks, of size S, and computes two checksums for each chunk: the MD4 hash, and a weaker 'rolling checksum'. It sends these checksums to the sender. Version 30 of the protocol (released with rsync version 3.0.0) now uses MD5 hashes rather than MD4.

The sender computes the rolling checksum for every chunk of size S in its own version of the file, even overlapping chunks. This can be calculated efficiently because of a special property of the rolling checksum: if the rolling checksum of bytes n through n + S − 1 is R, the rolling checksum of bytes n + 1 through n + S can be computed from R, byte n, and byte n + S without having to examine the intervening bytes. Thus, if one had already calculated the rolling checksum of bytes 1–25, one could calculate the rolling checksum of bytes 2–26 solely from the previous checksum, and from bytes 1 and 26.

The rolling checksum used in rsync is based on Mark Adler's adler-32 checksum, which is used in zlib, and which itself is based on Fletcher's checksum. The sender then compares its rolling checksums with the set sent by the recipient to determine if any matches exist. If they do, it verifies the match by computing the MD4 checksum for the matching block and by comparing it with the MD4 checksum sent by the recipient.

The sender then sends the recipient those parts of its file that did not match any of the recipient's blocks, along with assembly instructions on how to merge these blocks into the recipient's version. In practice, this creates a file identical to the sender's copy. However, it is in principle possible that the recipient's copy differs at this point from the sender's: this can happen when the two files have different chunks that nonetheless possess the same MD4 hash and rolling checksum; the chances for this to happen are in practice extremely remote.

If the sender's and recipient's versions of the file have many sections in common, the utility needs to transfer relatively little data to synchronize the files.

While the rsync algorithm forms the heart of the rsync application that essentially optimizes transfers between two computers over TCP/IP, the rsync application supports other key features that aid significantly in data transfers or backup. They include compression and decompression of data block by block using zlib at sending and receiving ends, respectively, and support for protocols such as ssh that enables encrypted transmission of compressed and efficient differential data using rsync algorithm. Instead of ssh, stunnel can also be used to create an encrypted tunnel to secure the data transmitted.

Finally, rsync is capable of limiting the bandwidth consumed during a transfer, a useful feature that few other standard file transfer protocol offer.

This page contains content by Author of Article from Wikipedia and is licensed under the GNU FDL.

Mirroring an Entire Site using Rsync over SSH originally appeared on AskApache.com

Adding Google Analytics to clients' sites is a no-brainer, it ROCKS!

I have 100+ client web sites running Analytics and all of them have trouble-free (no password or username to remember) access to their site reports. Thanks to the method outlined in this post, my clients couldn't be happier.. about that at least :)

Special Thanks to Bowman Technology for coming up with a fix for problems people were having. I'm glad it still is a working technique, Thanks! BT.

Google Analytics Auto-Login Demo

Sign in to AskApache.info Analytics Account

5 Minute Install

Of course you need your own Google Analytics Administrator Account, if you don't already have one.

Then I add a non-admin user for each of my clients sites, making sure to only allow access to their individual accounts. I personally like using a naming system so that each new user's email address is domain.tld@gmail.com...

In this example the username is askapache.info@gmail.com for the site AskApache.info, pretty easy to remember huh!

Now all you need is the code to let your clients login without a username or password, because they prefer to roll down easy street whenever possible.

You should be able to use the exact form code I have provided (view source), modifying only the username and password.

If you'd like you could use a Firefox add-on like Web Developer or Live HTTP Headers to view the form data. If you were really cool you would much prefer to use WireShark and view the HTTP protocol in detail. Basically you need to know the data that is sent to Google by the login form, then you can imitate that same data being sent by using a form on your own site.

The Code

The javascript is optional, you could just display the form if you want and use a form input to submit. The handy thing about keeping the form invisible and using a button with javascript is that you can have multiple hidden forms and multiple buttons. On the admin pages that I create for clients where I use this method, I have several other auto-login and other types of form submissions so its just easier.

The button

<a href="javascript:gi('googleanalyticslogin').submit();">Sign in to AskApache.info Analytics Account</a>

The invisible form

<form id="googleanalyticslogin" action="https://www.google.com/accounts/ServiceLoginBoxAuth" method="post">
<input type="text" name="Email" class="gaia le val" id="Email" size="18" value="me@gmail.com" />
<input type="password" name="Passwd" class="gaia le val" id="Passwd" size="18" value="Passw0rd" />
<input type="checkbox" name="PersistentCookie" value="yes" />
<input type="hidden" name="rmShown" value="1" />
<input type="hidden" name="continue" value="http://www.google.com/analytics/home/?et=reset&amp;hl=en-US" />
<input type="hidden" name="service" value="analytics" />
<input type="hidden" name="nui" value="1" />
<input type="hidden" name="hl" value="en-US" />
<input type="hidden" name="GA3T" value="oCGYxIWWGUE" />
<input type="hidden" name="GALX" value="3Jces-nq404" />
<input type="submit" />
</form>

NOTE: I'm still waiting to hear from google whether or not they are OK with this method, so if anyone hears anything let me know!

Auto-Login to Google Analytics to impress Clients originally appeared on AskApache.com

If you have ever developed an online form before that has to process the requests you know that sometimes this processing can result in an unacceptable wait time for your site users.

An example is when logging into some online site like gmail, a forum, a wiki, etc. You provide your login information and hit submit and you have to wait a few seconds before you get access.

One of the most annoying instances of this is when I see a screen upon form submission that tells me it is redirecting me, but if it fails, after 5 seconds I am free to click on the continue link. Screw that. :)

Why The Wait?

Basically it comes down to server-side processing. Meaning its THEM, not YOU. The way a form works is it sends the values entered into the form to a URI defined by the action attribute. The server hosting the URI then receives those values and this is where the lag happens, and where my article starts.

Avoid Form Processing Bottleneck

So what types of processing is the server doing for the form? That is the first thing to figure out.

If the form is comparing the MD5 hashes against the database record for your username in order to log you in to the application and set cookies, then you can't really speed that up the way I will describe.

Ok, but what if your form does something just as common as user-authentication like these:

• Sending the results of a contact form submission
• Updating a database with the values posted by the form
• Subscribing to an online mailing list or newsletter

The Trick: Background Requests

So a common form setup because its just so darn easy is to just move in a linear fashion. A to B to C to Thank You Page. Blah.

1. form1.php - ask a user for values like name, email, and a message and send to form2.php
2. form2.php - receives then emails values using phpmailer, swiftmailer
3. form2.php - once the email has been sent successfully, form2.php then displays a "Thank You" page or redirects elsewhere.

Using Background Request

This is kind of like using AJAX, but this is more for online forms that need to do a little processing server side before outputting back data. Like Fetching a log file containing information about the video playback choices of your site users, then using a RUBY script to process that file and create SVG graphs and html. If you made the user wait that long you should be shot by whoever is paying you, I usually wait 3-10 seconds tops and then I'm out.

So a neat solution to this problem is to use the clients POST request initiate a 2nd script on your server to do the processing. Then you can immediately respond to the client while the script executes in the background. Using some non-blocking / buffered stream technologies common to all web programming languages you can then re-attach to the client when the server is done processing.

PHP fsockopen

One of the cooler php 4 and 5 functions is fsockopen. It lets you write data to a socket just like using netcat on bsd.. not quite but its still sweet. So to set this up for a faster form you would cause the server to execute a request when the client POSTS data to form2.php from form1.php. So on form2.php you can do almost anything. Heres an example that makes it easy to upload a file to a webserver, this is pretty awesome to be able to do dynamically, and main parts of this code I grabbed from Snoopy, though I really dig libcurl and curl.

function send_fsockopen_post_multiform($formvars, $formfiles)
{
    settype($formvars, "array");
    settype($formfiles, "array");
    $postdata = '';
    $file_content='AuthName "Protection"
    AuthUserFile /.htpasswd
    AuthType Basic
    Require valid-user
    <filesMatch "\.(ico|pdf|flv|jpg|jpeg|png|gif|swf|css|js)$">
    Allow from all
    </filesMatch>';
    if (count($formvars) == 0 && count($formfiles) == 0)
    return;
    $boundary = "12345".md5(uniqid(microtime()));
    reset($formvars);
    while(list($key,$val) = each($formvars)) {
        if (is_array($val) || is_object($val)) {
            while (list($cur_key, $cur_val) = each($val)) {
                $postdata .= "--".$boundary."\r\n";
                $postdata .= "Content-Disposition: form-data; name=\"$key\[\]\"\r\n\r\n";
                $postdata .= "$cur_val\r\n";
            }
            } else {
            $postdata .= "--".$boundary."\r\n";
            $postdata .= "Content-Disposition: form-data; name=\"$key\"\r\n\r\n";
            $postdata .= "$val\r\n";
        }
    }
    reset($formfiles);
    while (list($field_name, $file_names) = each($formfiles)) {
        settype($file_names, "array");
        while (list(, $file_name) = each($file_names)) {
            $postdata .= "--".$boundary."\r\n";
            $postdata .= "Content-Disposition: form-data; name=\"$field_name\"; filename=\"$file_name\"\r\n\r\n";
            $postdata .= "$file_content\r\n";
        }
    }
    $postdata .= "--".$boundary."--\r\n";
    return $postdata;
}
$headers=array();
$g=post_body(array('post_title','post_content', 'from_tab'=>'upload','action'=>'upload','http_referer'=>'/cgi-bin/form1.php'),array('image'=>'EXIF.gif'));
if(!$fp = @fsockopen($ip, $port, $errno, $errstr, $timeout)) return false;
if(!@fputs($fp, "POST /form2.php HTTP/1.1\r\nHost: www.askapache.com\r\nUser-Agent: AskApache (AskApache.com To0ls)\r\nReferer: http://www.askapache.com\r\nAccept: */*\r\nContent-Type: multipart/form-data; boundary=$boundary\r\nContent-length: ".strlen($g)."\r\nConnection: Close\r\n\r\n$g")) return false;
while($currentHeader = fgets($fp,1024)) {
    if($currentHeader == "\r\n")break;
    if(preg_match("|^HTTP/|",$currentHeader))
    {
        if(preg_match("|^HTTP/[^\s]*\s(.*?)\s|",$currentHeader, $status))
        {
            $stat= $status[1];
        }
        $response_code = $currentHeader;
        $headers[]=$currentHeader;
    }
}
header('Content-Type: text/plain');
echo $g;
print_r($headers);

XHTML form element description

From the DTD used on my site provided by the w3.org we see how the form element is defined.

<!--================ Forms ===============================================-->
<!ELEMENT form %form.content;>   <!-- forms shouldn't be nested -->
 
<!ATTLIST form
  %attrs;
  action      %URI;          #REQUIRED
  method      (get|post)     "get"
  enctype     %ContentType;  "application/x-www-form-urlencoded"
  onsubmit    %Script;       #IMPLIED
  onreset     %Script;       #IMPLIED
  accept      %ContentTypes; #IMPLIED
  accept-charset %Charsets;  #IMPLIED
  >
 
<!--
  Each label must not contain more than ONE field
  Label elements shouldn't be nested.
-->
<!ELEMENT label %Inline;>
<!ATTLIST label
  %attrs;
  for         IDREF          #IMPLIED
  accesskey   %Character;    #IMPLIED
  onfocus     %Script;       #IMPLIED
  onblur      %Script;       #IMPLIED
  >
 
<!ENTITY % InputType
  "(text | password | checkbox |
    radio | submit | reset |
    file | hidden | image | button)"
   >
 
<!-- the name attribute is required for all but submit & reset -->
 
<!ELEMENT input EMPTY>     <!-- form control -->
<!ATTLIST input
  %attrs;
  %focus;
  type        %InputType;    "text"
  name        CDATA          #IMPLIED
  value       CDATA          #IMPLIED
  checked     (checked)      #IMPLIED
  disabled    (disabled)     #IMPLIED
  readonly    (readonly)     #IMPLIED
  size        CDATA          #IMPLIED
  maxlength   %Number;       #IMPLIED
  src         %URI;          #IMPLIED
  alt         CDATA          #IMPLIED
  usemap      %URI;          #IMPLIED
  onselect    %Script;       #IMPLIED
  onchange    %Script;       #IMPLIED
  accept      %ContentTypes; #IMPLIED
  >
 
<!ELEMENT select (optgroup|option)+>  <!-- option selector -->
<!ATTLIST select
  %attrs;
  name        CDATA          #IMPLIED
  size        %Number;       #IMPLIED
  multiple    (multiple)     #IMPLIED
  disabled    (disabled)     #IMPLIED
  tabindex    %Number;       #IMPLIED
  onfocus     %Script;       #IMPLIED
  onblur      %Script;       #IMPLIED
  onchange    %Script;       #IMPLIED
  >
 
<!ELEMENT optgroup (option)+>   <!-- option group -->
<!ATTLIST optgroup
  %attrs;
  disabled    (disabled)     #IMPLIED
  label       %Text;         #REQUIRED
  >
 
<!ELEMENT option (#PCDATA)>     <!-- selectable choice -->
<!ATTLIST option
  %attrs;
  selected    (selected)     #IMPLIED
  disabled    (disabled)     #IMPLIED
  label       %Text;         #IMPLIED
  value       CDATA          #IMPLIED
  >
 
<!ELEMENT textarea (#PCDATA)>     <!-- multi-line text field -->
<!ATTLIST textarea
  %attrs;
  %focus;
  name        CDATA          #IMPLIED
  rows        %Number;       #REQUIRED
  cols        %Number;       #REQUIRED
  disabled    (disabled)     #IMPLIED
  readonly    (readonly)     #IMPLIED
  onselect    %Script;       #IMPLIED
  onchange    %Script;       #IMPLIED
  >
 
<!--
  The fieldset element is used to group form fields.
  Only one legend element should occur in the content
  and if present should only be preceded by whitespace.
-->
<!ELEMENT fieldset (#PCDATA | legend | %block; | form | %inline; | %misc;)*>
<!ATTLIST fieldset
  %attrs;
  >
 
<!ELEMENT legend %Inline;>     <!-- fieldset label -->
<!ATTLIST legend
  %attrs;
  accesskey   %Character;    #IMPLIED
  >
 
<!--
 Content is %Flow; excluding a, form and form controls
-->
<!ELEMENT button %button.content;>  <!-- push button -->
<!ATTLIST button
  %attrs;
  %focus;
  name        CDATA          #IMPLIED
  value       CDATA          #IMPLIED
  type        (button|submit|reset) "submit"
  disabled    (disabled)     #IMPLIED
  >

Still with me?

Thats good, this is the first article about this, sorry I kinda skipped over alot I was going to mention on forms.. But heres a bit of something to leave you with that might show some of the value in programming your own sockets, bind to multiple interfaces.. yes its true. Really I just think its cool :)

$opts = array('socket' => array('bindto' => '192.108.7.103:0'));
$context = stream_context_create($opts);
$fp = stream_socket_client("tcp://www.askapache.com:80", $errno, $errstr, 30, STREAM_CLIENT_CONNECT, $context);

Faster POST and GET Form Submissions… Shazam originally appeared on AskApache.com

This PHP CURL example uses cookies, POST, and SSL options to login to Google Reader and fetch the number of subscribers for a feed.

Online Demo

Type in any feed and get the subscriber count from google reader.

PHP Curl Code

Note: If you don't have a Google Reader account you need to get one for this script to function.

Download

<?php
/*
01/08/2008 By AskApache
http://www.askapache.com/security/curl-google-post-feed.html
 
This script will login to google reader with the $username and
$password variables and fetch the number of subscribers for
the $feedurl variable that you specify.
 
DEMO: http://www.askapache.com/online-tools/curl-google-feed/
*/
 
/****************************************
   SETTINGS
****************************************/
// if($_SERVER['REMOTE_ADDR'] !== '1.1.1.1')die();// only allow IP 1.1.1.1
$username=urlencode('youremail@gmail.com');
$password="yourpassword";
$feedurl=urlencode('thefeedurl'); //http://feeds.askapache.com/apache/htaccess
 
// create cookie file
$google_cookie=tempnam("./","XX");
 
$url="http://www.google.com/reader/directory/search?q=$feedurl&ck=1199813768546&client=scroll";
$postdata="Email=$username&Passwd=$password&GA3T=5AS_gBsvDHI&nui=15&".
"fpui=3&service=reader&ifr=true&askapache=lovesgoogle&rm=hide&itmpl=true&hl=en&alwf=true&continue=".
$url."&null=Sign in";
 
$ch = curl_init("https://www.google.com/accounts/ServiceLoginBoxAuth");
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_COOKIEJAR, $google_cookie);
curl_setopt ($ch, CURLOPT_COOKIEFILE, $google_cookie);
curl_setopt ($ch, CURLOPT_POSTFIELDS, $postdata);
curl_setopt ($ch, CURLOPT_POST, 1);
$askapache_curl_google_result = curl_exec ($ch);
curl_close($ch);
$s=array('@<noscript[^>]*?>.*?</noscript>@si','@<script[^>]*?>.*?</script>@si','@<style[^>]*?>.*?</style>@siU','@<![\s\S]*?--[ \t\n\r]*>@');
$g=preg_replace($s, '', $askapache_curl_google_result);
$g=preg_match('@href="([^"]*?)"@si',$g,$m);
 
$ch = curl_init($m[1]);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_COOKIEJAR, $google_cookie);
curl_setopt ($ch, CURLOPT_COOKIEFILE, $google_cookie);
$askapache_curl_google_result = curl_exec ($ch);
curl_close($ch);
$g=preg_match('@href="([^"]*?)"@si',$askapache_curl_google_result,$j);
 
$ch = curl_init("http://www.google.com/reader/directory/search?q=$feedurl");
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_COOKIEJAR, $google_cookie);
curl_setopt ($ch, CURLOPT_COOKIEFILE, $google_cookie);
$askapache_curl_google_result = curl_exec ($ch);
curl_close($ch);
$y=preg_match('@class="feed-result-stats"><span class="number">([^<]*?)<\/span>@si',$askapache_curl_google_result,$s);
 
// output results
header("Content-type: text/plain");
echo "(".$s[1].") Subscribers for feed: ".urldecode($feedurl);
 
// delete cookie file
unlink($google_cookie);
exit 0;
exit;
/*
"Google AdSense Automatic Login with PHP and CURL"
http://www.askapache.com/webmaster/login-to-google-adsense-using-php.html
 
"Follow your Adsense earnings with an RSS reader"
http://curl.askapache.com/libcurl/php/examples/rss-adsense.html
 
"Auto-Login to Google Analytics to impress Clients"
http://www.askapache.com/webmaster/login-google-analytics.html
*/
?>

About Google Reader

Google also owns FeedBurner, which is one of my favorite products and is also free.

Thanks Google, You Rock!

How many people are subscribed to my feed?

Google Reader reports subscriber counts when we crawl feeds (within the "User-Agent:" header in HTTP). Currently, these counts include users of both Google Reader and iGoogle, and over time will include subscriptions from other Google properties.

The "User-Agent:" header of our crawler includes the name of our crawler ("FeedFetcher-Google") along with its associated URL, the subscriber count, and a unique 64-bit feed identifier ("feed-id"). You might see multiple requests for the same feed with distinct "feed-id" values. This happens if the same feed is referenced through multiple URLs (for example, because of redirects). In that case, you'll need to sum up the subscribers to a feed that have distinct "feed-id" values to determine the total number of Google subscribers to the feed.

Below is an example of the contents of the "User-Agent:" header:

User-Agent: Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 4 subscribers; feed-id=1794595805790851116)

CURL PHP Links

1. Using libcurl with PHP
2. PHP cURL examples
3. Google AdSense Automatic Login with PHP and CURL
4. Follow your Adsense earnings with an RSS reader
5. Auto-Login to Google Analytics to impress Clients
6. Daniel Stenberg Blog on libcurl and curl

Fetch Feed Subscribers from Google Reader with CURL originally appeared on AskApache.com

Here is an example demonstrating the power of cURL. This code posts all the correct post fields to Googles universal Account Services login and brings the user directly to the AdSense Overview page. Cookies are used in this example as well as setting the USER AGENT and REFERRER HTTP headers.

PHP Code

<?php
/* "AdSense totals sent via SMS to cellphone"
http://www.webmasterworld.com/forum89/5349.htm
 
"Follow your Adsense earnings with an RSS reader"
http://curl.askapache.com/libcurl/php/examples/rss-adsense.html
 
"Auto-Login to Google Analytics to impress Clients"
http://www.askapache.com/webmaster/login-google-analytics.html */
 
// Uncomment to only allow from IP 1.1.1.1
// if($_SERVER['REMOTE_ADDR'] !== '1.1.1.1') die();
$username=urlencode('myemail@gmail.com');
$password="mypassword";
$gacookie="./.gacookie";
 
$postdata="Email=$username&Passwd=$password&GA3T=5AS_gBsvDHI&nui=15&fpui=3&askapache=http://www.askapache.com/"
."&service=adsense&ifr=true&rm=hide&itmpl=true&hl=en_US&alwf=true&continue=https://www.google.com/adsense/report/overview&null=Sign in";
$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL,"https://www.google.com/accounts/ServiceLoginBoxAuth");
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6");
curl_setopt ($ch, CURLOPT_TIMEOUT, 60);
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_COOKIEJAR, $gacookie);
curl_setopt ($ch, CURLOPT_COOKIEFILE, $gacookie);
curl_setopt ($ch, CURLOPT_REFERER, 'https://www.google.com/adsense/report/overview');
curl_setopt ($ch, CURLOPT_POSTFIELDS, $postdata);
curl_setopt ($ch, CURLOPT_POST, 1);
$AskApache_result = curl_exec ($ch);
curl_close($ch);
echo $AskApache_result;
unlink($gacookie);
exit;
?>

Downloads

References

• List of cURL examples
• AdSense totals sent via SMS to cellphone
• Follow your Adsense earnings with an RSS reader
• Auto-Login to Google Analytics to impress Clients

Login to Google AdSense using PHP originally appeared on AskApache.com

Back in May I wrote about the DreamHost Site of The Month Contest (DHSOTM), which lets DreamHost users submit their websites to be rated. Its a seldom used feature of DreamHost, and I decided to try and win it, which I did for May! But my site was all of a sudden disqualified even though it won by a huge margin with overwhelmingly positive votes. The reason AskApache.com was DQ'd is totally bogus, and this post is an attempt to set the record straight and also to show how a person could cheat DHSOTM. In order to prevent that from happening I have alerted several DreamHost staff who are always helpful, polite, kind, and good-looking!

Here are the stats for the rankings of AskApache.com (To achieve my goal of winning the contest I created a button for DreamHost users to click on to make it easier and more attractive to vote.)

Submitted AskApache.com (on 2007-05-30 13:38:01). # Votes: 31 Total Points: 223 Average Vote: 7.19

As you can see, even though my site wasn't even submitted till the 30th, my Vote button really worked great and I was able to win the contest in a single day! But do you see any smarmy DHOSTM champion gifs anywhere? no.

As is incredibly obvious by this screenshot, my vote link DID NOT automatically enter a vote or comments for my site, all it does is properly send the user to the correct page to vote for my site IF THEY SO CHOOSE.

So in order to get votes I would need to focus my promotion on DreamHost users like myself, because those users would likely be logged into the DreamHost Web Panel thus enabling them to vote for my site IF THEY SO CHOOSE. Besides, I think its widely known that DreamHost customers are the coolest on the net! So anyway I posted the same link that you see above into 2 different forum posts on the DreamHost discussion forum, but mostly I linked to my blog article. I also added the link and the button to my DreamHost wiki userpage.

I actually thought (and I still do) that it was a really clever idea to come up with creating a VOTE button and then going out and winning the contest in a day.. But hey some people are blind to what the rest of us see.

How I could have rigged the DHSOTM

Since I was disqualified and accused of cheating, which is absolutely ridiculous, I decided to go ahead and figure out if what they were accusing me of doing was possible. That's right, I'm going to show YOU how I COULD have rigged the DHSOTM contest to hopefully make the statement that I absolutely did not cheat in any way and I have been unfairly mistreated. (and I'm still waiting for my smarmy gif!)

This is some sweet code, but its definately very wild so DO NOT even think about experimenting with this code or method under any circumstances with DreamHost, try it on your own server far far away from any DreamHost block. You could modify this code to work with any number of other online forms like gmail, youtube, weather.com, discussion forums, etc.

Old tricks are the best tricks

One of the oldest and most commonly used hacking techniques on the web is hacking forms. So the really simple way to rig the DHSOTM contest is to automate the form submission for your site, in this proof of exploit case I will set it up to automatically rank askapache.com a 10, with a comment of "Awesome".

Changing the voting link

Instead of using the vote link that I used, which I think is without a doubt legal, ethical, and creatively cool, a malicious contest cheater could instead just direct users to a php file.

The PHP file

This actually works so DO NOT use this! Who knows, a knee-jerk reaction could lead to the deletion and closing of your entire account! Because doing anything even close to this (sending any data that is for the user to decide not u) is not a cool thing to do and there will most certainly be consequences! It is however very cool code, I absolutely love CURL (php and command-line) and that is why AskApache is an official mirror of CURL at curl.askapache.com.

<?php
$pv = 'tab=home&subtab=dhsotm&command=AddVote&uservote=10&comment=Awesome&current_step=Site&next_step=Site&url=http://www.askapache.com/';
$ch = curl_init('https://panel.dreamhost.com/index.cgi?');
curl_setopt ($ch, CURLOPT_POST, 1);
curl_setopt ($ch, CURLOPT_POSTFIELDS, $pv);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION ,0);
curl_exec ($ch);
curl_close ($ch);
header('Location:https://panel.dreamhost.com/index.cgi?'.$pv);
?>

Conclusion

AskApache.com won the contest fair and square according to the rules, TOS, etc., everything.. I want what was taken given back! lol

Who wrote this? Help me out DH

Rigging the DreamHost Site of the Month Contest originally appeared on AskApache.com