<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>AskApache &#187; Search Results  &#187;  pipelining</title>
	<atom:link href="http://www.askapache.com/search/pipelining/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.askapache.com</link>
	<description>Advanced Web Development</description>
	<lastBuildDate>Thu, 26 Apr 2012 11:29:28 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>PortaPutty Auto-Reconnecting SSH Tunnels on an Encrypted TrueCrypt Portable USB Key w GPG</title>
		<link>http://www.askapache.com/hacking/ssh-tunnels-truecrypt-gpg.html</link>
		<comments>http://www.askapache.com/hacking/ssh-tunnels-truecrypt-gpg.html#comments</comments>
		<pubDate>Tue, 23 Feb 2010 10:11:11 +0000</pubDate>
		<dc:creator>AskApache</dc:creator>
				<category><![CDATA[Hacking]]></category>

		<guid isPermaLink="false">http://askapache.com/?p=4001</guid>
		<description><![CDATA[<p><a class="IFL" id="id6" href="http://www.askapache.com/hacking/ssh-tunnels-truecrypt-gpg.html"></a>Ok I just came back up to write the intro.. I'm trying to keep it short to avoid getting bogged down by the coolness of each step.  Here is what goes on.   When I logon to my XP machine at work, I bring my usb key and plug it in first.  On logging a window pops up first and it's a password prompt to mount my encrypted drive leonardo.  It also checks a keyfile that is located on my usb key, but all I do now is type in my password.  That causes my encrypted folder to be accessible to me like a normal drive, and it autoruns a startup batch file. <br /><br />The batch file causes <strong>Portable</strong> versions of Firefox (<em>all my bookmarks, my settings</em>) to load, and launches <strong>Portable</strong> Mozilla Thunderbird (IMAP makes this work well), which is my favorite program (<em>great GPG features and open-source!</em>).  Also Some Adobe CS4 software is loaded from the hard drive, like DreamWeaver.  In the background, a service we created executes a PortaPuttY plink command to create forwarded tunnels from various remote servers and accounts, all using key-based encryption.  These tunnels are automatically reconnected if they are disconnected, meaning you can use a socks 5 if you want or even better!<br class="C" /></p>
<p><strong>Part 1 of 5</strong></p>]]></description>
			<content:encoded><![CDATA[<p><a href="http://www.askapache.com/hacking/ssh-tunnels-truecrypt-gpg.html"></a><a href="http://www.askapache.com/hacking/ssh-tunnels-truecrypt-gpg.html"><cite>AskApache.com</cite></a></p><p><a id="id6" class="IFL" href="http://www.tigerdirect.com/applications/SearchTools/item-details.asp?EdpNo=4329073&CatId=3786" title="This ones over $400 dollars! 128GB!"></a>Ok I just came back up to write the intro.. I'm trying to keep it short to avoid getting bogged down by the coolness of each step.  Here is what goes on.   When I logon to my XP machine at work, I bring my usb key and plug it in first.  On logging a window pops up first and it's a password prompt to mount my encrypted drive leonardo.  It also checks a keyfile that is located on my usb key, but all I do now is type in my password.  That causes my encrypted folder to be accessible to me like a normal drive, and it autoruns a startup batch file.  The batch file causes <strong>Portable</strong> versions of Firefox (<em>all my bookmarks, my settings</em>) to load, and launches <strong>Portable</strong> Mozilla Thunderbird (IMAP makes this work well), which is my favorite program (<em>great GPG features and open-source!</em>).  Also Some Adobe CS4 software is loaded from the hard drive, like DreamWeaver.<br class="C" /></p>

<p>The batch file also runs PortaPuttY plink to create forwarded tunnels from various remote servers and accounts, all using key-based encryption.  This includes dynamic SOCKS 4/5 tunnels, VPN tun device tunnels, and of course the basic SSH port forwarding tunnels that are so powerful.  These tunnels are automatically reconnected if they are disconnected, using simple windows builtin command-line tools.  And believe me it was not easy to figure out how to make this all work using plink ( essentially the same as putty minus the gui ), I literally had to use almost all of my Windows kung fu to finally end up with this.</p>

<h2>Using MyEnTunnel</h2>
<p>Initially I was using the <a href="http://nemesis2.qx.net">MyEnTunnel</a> program combined with a custom windows batch install script I wrote to handle the tunnels.<br /><br />The tunnels are very important to making things easy while improving security.  It's not easy to understand at first, but basically it means <strong>you can now connect to ANY IP address:port as if you were on that very machine connecting to localhost, like if you pinged yourself!</strong>.  The result is any traffic you want is now encrypted, and you can set up your servers to only accept connections from localhost, which could save you tons of memory, bandwidth, and security attack vectors to think about.   So I configure everything to use these tunnels as proxies, like Mozilla Thunderbird and Chrome, Firefox, Pidgeon, all portable versions and running from my encrypted usb drive.</p>

<p>This means you can walk into my house with that usb key, plug into any computer here, and surf the web/check your emails all across SSH... I know for a fact <strong>I wouldn't be able to snoop that</strong> traffic!  There is a lot of exciting things going on around here, new servers and all.. Its going to take a couple more posts for me to finish this up, enjoy the article and comment.</p>



<h2>Buy a couple USB Mini Drives</h2>
<p>The first thing to do, is purchase a USB thumb drive..   My favorite store, <a href="http://www.tigerdirect.com/applications/category/category_slc.asp?Recs=30&amp;Nav=|c:379|lp:0.01:hp:24.99|&amp;Sort=4" title="Highly recommended, this is not an affiliate link">TigerDirect.com</a>, has over 104 tiny usb drives for under $24.. I've used them since the late 90's.</p>
<p>I bought some 4GB PNY's the size of a fingernail at a gas station and they are amazing, way faster than say a dvd drive.  Just try to do some research of the differences between the 16GB vs the $4 1GB drives.. You want speed because the whole drive will be encrypted.   <em>If you can afford the super excellent and crazy fast ones, hey send me one!</em> Buying cheap means you can buy 3 or 4 so you can always have backups.  This device will make you Internationally mobile, untethered from a box, maybe getting some work done at a cafe in Florenze, or at a beach hotel in Miami.  Keep dreaming, but that is more possible with a better organized system.</p>

<h3>Backup the USB Drive</h3>
<p>You only need to know 1 way that works, there are several.  The way I do backups is to copy the entire disk image of the usb, that way I can always access it in case of usb key failure, which does happen.  Free software like <a href="http://clonezilla.org/">CloneZilla live CD</a> with its crazy cluster computing power, or Self Image, which is free for both linux and Windows.  And you could never go wrong with <a href="http://www.symantec.com/norton/ghost">GHOST</a>, one of the first to make mega bucks in the market.. it's some seriously impressive software <em>but not open-source</em>.  Even easier for some is to just set a cron job for dd to pipe the entire drive image to a remote computer using netcat, or sshfs, or curlftpfs, or just <strong>simple ssh</strong> like below.  Once setup (without stupid, bulky, dangerous software), the files on your encrypted usb don't change often, otherwise I would want to sync a backup to happen automatically every X number of logins or days (<em>test logfile time in bash_profile?</em>)..</p>

<h4>SSH Back-ups To Remote Server</h4>
<p>Files and data on your drives slow it down tremendously, meaning a web server storing backups locally is slower than one storing them externally.</p>
<p>Notice how much safer this command is by optimizing both the <a href="http://www.askapache.com/linux/optimize-nice-ionice.html">CPU and DISK I/O</a>..  Though it's much smarter to create a new separate ssh user, one with no shell and a passwordless safer key-based encryption.  Then in your /etc/security/limits.conf file or your initscript you can cause that user to have <code>nice -19</code> and <code>ionice -c2 -n7</code> priority set all the time automatically, since sshd, compression, and disk writing are this accounts only job.  turboslow is an alias defined in a ssh_config file so you don't have to type the host, port, and settings each time.</p>
<pre>#
# much better ways to do this on google!!!!!!!
#
ionice -c2 -n7 nice -n 19 dd if=/dev/sdb2 bs=1k conv=sync,noerror | gzip -c | ssh turboslow "dd of=sdb2.gz bs=1k"</pre>
<p>Note that you may decide it would be better to configure the ssh connection to a less CPU intensive algorithm, perhaps even <em>protocol 1</em> and <em>DES</em>.  That's perfectly alright, but the tradeoff is that the encryption can be broken much quicker, and so you would have to implement a cron job to create new keys on both ends of the tunnel every few hours.. It's really not a big deal to setup, kind of sweet way to use key-based encryption.  Also, important files ( those containing passwords, any database ) are encrypted before transport using private GPG keys, which don't need to be changed.  The other thing to think about too is only letting your main PC send/write on the backup host, so the backup host is only authorized to rx and can never login back to yours. </p>
<p>Hey! the Internet is a dangerous place you better believe it!  And it's only going to get more interesting with cloud computing's breakthrough's... More people who know they're way around... I can always use an extra server, I'd love to expand my network another node without having to pay for it (free cloud computing?), so make sure your servers are locked up strenuously.  Not super perfect, just a little unique or creative in your defense to avoid any coming super-worm's that may be employing vast arsenals of the deadliest attack-engines like metasploit..  Scarry rumors.</p>

<h4>Compression Speeds: PBZip2, Rzip, Lzop, Gzip</h4>
<p>Probably the fastest is to use rsync over ssh, which is what I'm doing, since the algorithms used by rsync are much faster and safer.  <a href="http://www.askapache.com/security/mirror-using-rsync-ssh.html">Rsync also lets you specify a compression program</a>, so depending on your machine you will want pbzip2 (for multi processors) or rzip which are the 2 fastest I know of, though I have had some reliability issues with rzip for gigabyte transfers.  Pbzip2 is amazing, blew me away the first time being 8x faster (8 CPUs) then <strong>anything</strong>.  You can get it and compile a static binary for your thumb drive if want at <a href="http://compression.ca/pbzip2/">Parallel BZIP2 (PBZIP2)</a>. Heavy code, re: this note by Jeff Gilchrist</p>
<quote><strong>NOTE</strong>: If you are looking for a parallel BZIP2 that works on cluster machines, you should check out <strong>MPIBZIP2</strong> <strong>which was designed for a distributed-memory message-passing architecture</strong>.</quote>
<pre>tar cpf "$G" --use-compress-prog=pbzip2 ./</pre>
<p><a href="http://uploads.askapache.com/2010/02/pbzip2.gif"><img src="http://uploads.askapache.com/2010/02/pbzip2.gif" alt="Parallel BZIP2 (PBZIP2)" title="pbzip2" class="size-medium wp-image-4002" /></a></p>


<h4>Benchmarking for Performance</h4>
<p>Finally a couple tips, you should get an idea what the device can do, format it a few times for linux and test it on windows, and vice versa.. Some drives are too small or too old and can only support fat32 filesystems on winblows, you DO NOT want fat32 because this drive is going to be 100% encrypted and then 100% transparently decrypted as you use it,
<pre># note this is 512MB
dd if=/dev/sda1 of=/dev/null bs=512 count=1000000
512000000 bytes (512 MB) copied, 5.16588 s, 99.1 MB/s</pre>








<h2>Part II:  Encrypted AutoRunning USB Key with TrueCrypt</h2>
<p>Now this section anyone can do, it's so easy on Windows.   What I'm going to show you how to do is get setup the right way super-fast.  There are many ways to use TrueCrypt, it's one of the nicest built software programs's I've ever used... Sadly, it is not licensed open-source, and that is often a deal-breaker for security-conscious folks or anti-pirate anarchists.  From the very helpful TrueCrypt web site:</p>

<blockquote cite="">
<ul type="disc">
            <li>Creates a <strong>virtual encrypted disk</strong> within a file and mounts  it as a real disk. </li>
            <li>Encrypts an<strong> entire partition or storage device</strong> such as USB flash drive or hard drive.</li>
            <li>Encrypts a <strong>partition or drive where Windows is installed</strong> (<a href="http://www.truecrypt.org/docs/?s=system-encryption">pre-boot authentication</a>).</li>
            <li>Encryption is <a href="http://www.truecrypt.org/docs/"><strong>automatic</strong>, <strong>real-time</strong> (on-the-fly) and <strong>transparent</strong></a>.</li>
            <li> <a href="http://www.truecrypt.org/docs/?s=parallelization">Parallelization</a> and <a href="http://www.truecrypt.org/docs/?s=pipelining">pipelining</a> allow data to be read and written as fast as if the drive was not encrypted.</li>
            <li>Provides <strong><a href="http://www.truecrypt.org/docs/?s=plausible-deniability">plausible deniability</a></strong>, in case an adversary  forces you to reveal the password: <strong><a href="http://www.truecrypt.org/hiddenvolume">Hidden volume</a></strong> (steganography) and <strong><a href="http://www.truecrypt.org/docs/?s=hidden-operating-system">hidden operating system</a></strong>.</li>
            <li><a href="http://www.truecrypt.org/docs/?s=encryption-algorithms">Encryption algorithms</a>: <a href="http://www.truecrypt.org/docs/?s=aes">AES-256</a>, <a href="http://www.truecrypt.org/docs/?s=serpent">Serpent</a>, and <a href="http://www.truecrypt.org/docs/?s=twofish">Twofish</a>.  Mode of operation: <a href="http://www.truecrypt.org/docs/?s=modes-of-operation">XTS</a>.</li>
        </ul>

</p>
</blockquote>





<h2>Further Reading</h2>
<ul>
<li><a href="http://ultra.ap.krakow.pl/~bar/DOC/ssh_backup.html">Network File Copy using SSH</a></li>
<li>Check out the trunk version of PuTTY:<code>~ svn co svn://svn.tartarus.org/sgt/putty</code>
</ul>





<hr class="C" />
<p>The real fun doesn't start till all the automation starts, automating all of that from a couple batch files I wrote, one click setup.  Kind of like building your own knoppix for when you have to use Windows.  To begin this tutorial, setup a truecrypt traveller setup on your usb and also install the portaputty package onto the usb.  You do this by creating a 3GB or whatever file on the usb and then mounting that file like you would mount an iso file.   I will show the Windows Batch file I use and the tricks with Windows Volume names and how to consistently make it all work.   Then we will setup MyEnTunnel with a customized batch file that forces all puttys to use portaputty (<code>sweet hack stolen from sysinternals pagedefrag tool</code>).<strong>Stay Tuned!</strong></p><p><a href="http://www.askapache.com/hacking/ssh-tunnels-truecrypt-gpg.html"></a><a href="http://www.askapache.com/hacking/ssh-tunnels-truecrypt-gpg.html">PortaPutty Auto-Reconnecting SSH Tunnels on an Encrypted TrueCrypt Portable USB Key w GPG</a> originally appeared on <cite>AskApache.com</cite> </p>]]></content:encoded>
			<wfw:commentRss>http://www.askapache.com/hacking/ssh-tunnels-truecrypt-gpg.html/feed</wfw:commentRss>
		<slash:comments>8</slash:comments>
		</item>
		<item>
		<title>Serve External Javascript Files locally for Increased Speed</title>
		<link>http://www.askapache.com/javascript/serve-external-javascript-files-locally-for-increased-speed.html</link>
		<comments>http://www.askapache.com/javascript/serve-external-javascript-files-locally-for-increased-speed.html#comments</comments>
		<pubDate>Wed, 01 Oct 2008 06:55:46 +0000</pubDate>
		<dc:creator>AskApache</dc:creator>
				<category><![CDATA[Javascript]]></category>

		<guid isPermaLink="false">http://www.askapache.com/?p=1542</guid>
		<description><![CDATA[<p><a class="IFL hs hs16" href="http://uploads.askapache.com/2008/10/google-external-javascript.png" title="Screenshot of bash shell script for downloading externally hosted javascript files"></a>One way I speed up AskApache.com is by downloading external third-party javascript files to host on my own server instead of externally.  In addition to the obvious speed boost, this lets you configure the caching and compression settings for the files.<br class="C" /></p>]]></description>
			<content:encoded><![CDATA[<p><a href="http://www.askapache.com/javascript/serve-external-javascript-files-locally-for-increased-speed.html"></a><a href="http://www.askapache.com/javascript/serve-external-javascript-files-locally-for-increased-speed.html"><cite>AskApache.com</cite></a></p><p><a class="IFL hs hs16" href="http://uploads.askapache.com/2008/10/google-external-javascript.png" title="Screenshot of bash shell script for downloading externally hosted javascript files"></a>One of the ways I speed up AskApache.com is by downloading all the external javascript files to my server and then serve them from my own server instead of the external site.  Currently I'm using this method to serve the <a href="http://www.google-analytics.com/ga.js">Google Analytics</a> javascript file, and the <a href="http://edge.quantserve.com/quant.js">Quantcast</a> javascript file.<br class="C" /></p>


<h2>Speed Benefit</h2>
<p>The speed benefit occurs because normally a site visitor has to perform a DNS query for both the google-analytics.com and quantcast.com servers, create a connection to each server, and then download the files.  By hosting these scripts on your own server you remove all the extra DNS queries, and by hosting all the files on your single server you enable advanced HTTP 1/1 protocol features like Pipelining the requests, which means a single connection for multiple files.</p>


<h2>Cache and Compression Control</h2>
<p>Serving these javascript files from your own server gives you complete control over the caching and compression settings of those files, which may not seem like a big deal... unless you know about the advanced caching and compression techniques available to make your site super-fast.</p>

<h3>Caching Control</h3>
<p>By using the <a href="http://www.askapache.com/htaccess/mod_rewrite-fix-for-caching-updated-files.html">mod_rewrite cache hack</a> like I do, you can make sure that a fresh version of the <code>.js file</code> is used by your visitors.</p>

<h3>Compression Control</h3>
<p>You can also specify advanced compression for these files, which could mean using Apache's mod_deflate or mod_gzip modules, or you can even use a compression tool like Dojo's Shrinksafe, Packer, JSMin, or YUICompressor to further compress the javascript files.</p>


<h2>Automated Downloading with Crontab</h2>
<p>Using crontab I tell my web server to run this shell script every day, to make sure I have updated versions of all the files.  Here's the crontab entry I use to run this shell script every day.</p>
<pre>@daily /bin/sh /home/cron-scripts/grab_javascripts.sh 2&gt;&amp;1 &amp;&gt;/dev/null</pre>


<h2>Shell Script to Download JS</h2>
<p>I created a simple Bash shell-script to download the javascript files.  All you need to do is modify the variables to save to your directory and send your site as the Referring URL and you are good to go.  You can add as many files as you want by adding them to the JSFILE array.</p>
<p>You can also download the shell-script: <a href='http://uploads.askapache.com/2008/10/grab_javascripts.sh'>grab_javascripts.sh</a></p>
<pre>#!/bin/bash
# 2008-09-30
umask 022
VER="2.0"
&nbsp;
### Set as the referring url for downloads
SITE=http://www.askapache.com/
&nbsp;
### Directory to save the downloaded files
JSD=$HOME/j/
&nbsp;
### The files to download
JSFILE[0]=http://www.google-analytics.com/ga.js
&nbsp;
### run script with args to turn on debugging
[[ $# -ne 0 ]] &amp;&amp; set -o xtrace &amp;&amp; set -o verbose
&nbsp;
### SHELL OPTIONS
set +o noclobber # allowed to clobber files
set +o noglob # globbing on
set -e # abort on first error
shopt -s extglob
&nbsp;
#--=--=--=--=--=--=--=--=--=--=--#
# pt
#==-==-==-==-==-==-==-==-==-==-==#
function pt(){
  case ${1:-d} in
   i) echo -e "${C6}=&gt; ${C4}${2} ${C0}"; ;;
   m) echo -en "\n\n${C2}&gt;&gt;&gt; ${C4}${2} ${C0}\n\n"; ;;
   *) echo -e "\n${C8} DONE ${C0} \n"; ;;
  esac
}
&nbsp;
#=# CATCH SCRIPT KILLED BY USER
trap &#039;kill -9 $$&#039; SIGHUP SIGINT SIGTERM
&nbsp;
#=# MAKE MAIN SCRIPT NICE AS POSSIBLE SINCE IT DOESNT DO MUCH
renice 19 -p $$ &amp;&gt;/dev/null
&nbsp;
#=# TURNS ON COLORING ONLY FOR TERMS THAT CAN SUPPORT IT
C="\033[";C0=;C1=;C2=;C3=;C4=;C5=;C6=;C7=;C8=;C9=;
  C0="${C}0m";C1="${C}1;30m";C2="${C}1;32m";C3="${C}0;32m";C4="${C}1;37m"
  C5="${C}0;36m";C6="${C}1;35m";C7="${C}0;37m";C8="${C}30;42m";C9="${C}1;36m";
clear
echo -e "${C1} __________________________________________________________________________ "
echo -e "| ${C2}             ___       __    ___                 __             ${C1}         |"
echo -e "| ${C2}            / _ | ___ / /__ / _ | ___  ___ _____/ /  ___        ${C1}         |"
echo -e "| ${C2}           / __ |(_-&lt;/  &#039;_// __ |/ _ \/ _ \`/ __/ _ \/ -_)       ${C1}         |"
echo -e "| ${C3}          /_/ |_/___/_/\_\/_/ |_/ .__/\_,_/\__/_//_/\__/        ${C1}         |"
echo -e "| ${C3}                               /_/                              ${C1}         |"
echo -e "|                                                                          |"
echo -e "|                 ${C4} LOCAL JAVASCRIPT FILES SCRIPT Version ${VER} ${C1}              |"
echo -e "${C1} __________________________________________________________________________ ${C0} \n\n"
&nbsp;
#=# BUILD INCOMING DIRS
[[ ! -d "${JSD}" ]] &amp;&amp; pt m "BUILDING DIRS" &amp;&amp; mkdir -p $JSD &amp;&gt;/dev/null &amp;&amp; pt i "CREATED $JSD" &amp;&amp; pt
&nbsp;
#=# DOWNLOAD JAVASCRIPT FILES
cd $JSD &amp;&amp; pt m "DOWNLOADING JAVASCRIPT FILES"
for theurl in "${JSFILE[@]}"; do
pt i "${theurl}"
curl -m 60 --connect-timeout 10 --retry 10 --retry-delay 180 -s -S -L -e &#039;${SITE}&#039; -A &#039;Mozilla/5.0&#039; -O "${theurl}"
done
pt &amp;&amp; cd $OLDPWD
&nbsp;
######## curl options
# -S Show error. With -s, make curl show errors when they occur
# -s Silent mode. Don&#039;t output anything
# -e  Referer URL (H)
# -H &lt;line&gt; Custom header to pass to server (H)
# -L  Follow Location: hints (H)
# -A &lt;string&gt; User-Agent to send to server (H)
# -m &lt;seconds&gt; Maximum time allowed for the transfer
# --connect-timeout &lt;seconds&gt; Maximum time allowed for connection
# --globoff  Disable URL sequences and ranges using {} and []
# -O  Write output to a file named as the remote file
# --retry &lt;num&gt;  Retry request &lt;num&gt; times if transient problems occur
# --retry-delay &lt;seconds&gt; When retrying, wait this many seconds between each
# --retry-max-time &lt;seconds&gt; Retry only within this period
#############################################################################
&nbsp;
exit 0</pre><p><a href="http://www.askapache.com/javascript/serve-external-javascript-files-locally-for-increased-speed.html"></a><a href="http://www.askapache.com/javascript/serve-external-javascript-files-locally-for-increased-speed.html">Serve External Javascript Files locally for Increased Speed</a> originally appeared on <cite>AskApache.com</cite> </p>]]></content:encoded>
			<wfw:commentRss>http://www.askapache.com/javascript/serve-external-javascript-files-locally-for-increased-speed.html/feed</wfw:commentRss>
		<slash:comments>5</slash:comments>
		</item>
		<item>
		<title>Fsockopen Power Plays</title>
		<link>http://www.askapache.com/php/fsockopen-socket.html</link>
		<comments>http://www.askapache.com/php/fsockopen-socket.html#comments</comments>
		<pubDate>Wed, 02 Jul 2008 11:42:56 +0000</pubDate>
		<dc:creator>AskApache</dc:creator>
				<category><![CDATA[PHP]]></category>

		<guid isPermaLink="false">http://www.askapache.com/?p=1046</guid>
		<description><![CDATA[<p><a class="IFL hs hs17" rel="lb" href='http://www.askapache.com/php/fsockopen-socket.html' title="Fsockopen Power"></a><strong>PHP's <a href="http://php.net/manual/en/function.fsockopen.php">fsockopen</a> function lets you open an Internet or Unix domain socket connection for connecting to a resource, and is one of the most powerful functions available in the php language.</strong><br class="C" /></p>]]></description>
			<content:encoded><![CDATA[<p><a href="http://www.askapache.com/php/fsockopen-socket.html"></a><a href="http://www.askapache.com/php/fsockopen-socket.html"><cite>AskApache.com</cite></a></p><p><a class="IFL hs hs17" rel="lb" href='http://uploads.askapache.com/2008/07/fsockopen-lightning.jpg' title="Fsockopen Power"></a><strong>PHP's function <a href="http://php.net/manual/en/function.fsockopen.php">fsockopen</a> lets you open an Internet or Unix domain socket connection for connecting to a resource, and is one of the most powerful functions.</strong>  fsockopen could be described as creating a direct link to the wire connected to a resource, which means you can send any information (<em>EBCDIC, ASCII, Hex, C arrays, Raw</em>) directly to the target server.<br class="C" /></p>

<h2>A Socket is like /dev/null</h2>
<p>In unix you can send anything to the <code>/dev/null</code> device, <em>for Windows think Recycle Bin</em>, and likewise you can send anything to a socket created with fsockopen.  I've seen fsockopen code that sends custom exploits to cisco routers, including being used by the metasploit framework.  I've seen fsockopen telnet emulation, smtp/pop3 login, and a lot of other advanced raw networking that is exciting for me see.</p>

<h3>Some Definitions for Fsockopen</h3>
<dl>
<dt><a href="http://www.askapache.com/glossary/#client" title="client">client</a></dt>
<dd>A program that establishes connections for the purpose of sending requests.</dd>
<dt><a href="http://www.askapache.com/glossary/#server" title="server">server</a></dt>
<dd>An application program that accepts connections in order to service requests by sending back responses.</dd>
</dl>

<h3>Simple Socket Explantion</h3>
<p>A web server host listens on TCP port 80.  When a client host wishes to view a resource on the web server, it establishes a TCP connection with the server host by opening a socket to send the request for the resource.  When the connection is established, the client and server exchange requests and responses (respectively) until the connection is closed or aborted.</p>


<h2>HTTP and fsockopen</h2>
<p><a class="IFR" href='http://www.askapache.com/php/speedy-form-post.html' title='Snoopy Fsockopen HTTP Class for PHP'><img src='http://uploads.askapache.com/2008/02/snoopy-fsockopen.thumbnail.png' alt='Snoopy Fsockopen HTTP Class for PHP' title="snoopy fsockopen.thumbnail php" /></a>The <a href="http://snoopy.sourceforge.net/">Snoopy</a> class is bundled with WordPress distributions and uses fsockopen to achieve most of its cool features.  WordPress core, plugins, and other included files and classes also use the fsockopen function to communicate via HTTP.<br class="C" /></p>



<h2>Fsockopen Examples</h2>
<p><a rel="lb" class="IFL" href='http://uploads.askapache.com/2008/07/fsockopen-warning.jpg'><img src="http://uploads.askapache.com/2008/07/fsockopen-warning-200x181.jpg" alt="fsockopen warning" title="fsockopen warning" width="100" height="91" /></a>Note the warning sign, fsockopen is dangerous in the sense that you can crash your server, perform a DOS against your own server or other site, use up all your servers available sockets and fd descriptors, use up your bandwidth, etc.. Shouldn't be a problem unless you are being malicious or careless.<br class="C" /></p>
<p>Here are some BOSS fsockopen functions I hacked together yesterday for use in my <a href="http://wordpress.org/extend/plugins/askapache-crazy-cache/">AskApache Crazy Cache WordPress Plugin</a>.  I've used code and ideas from 100's of authors, projects, and docs to try to make this the very best I can.</p>

<h3>Intro</h3>
<p>This is a working example employing as many of the best-practices, tips, and tricks for using fsockopen on remote streams that I could find.</p>
<pre>&lt;?php
// max time for script execution
if(!@defined(&#039;AA_MAX_TIME&#039;)) define(&#039;AA_MAX_TIME&#039;,  60);
&nbsp;
// max time for socket reads
if(!@defined(&#039;AA_RECV_TIME&#039;)) define(&#039;AA_RECV_TIME&#039;, 30);
&nbsp;
// max time for socket connect
if(!@defined(&#039;AA_CONN_TIME&#039;)) define(&#039;AA_CONN_TIME&#039;, 5);
&nbsp;
// linebreak
if(!@defined(&#039;AA_LF&#039;)) define(&#039;AA_LF&#039;, chr(13).chr(10));
&nbsp;
// ignore TCP RST i.e. browser stop button
@ignore_user_abort(1);
&nbsp;
// set the script execution time
@set_time_limit(AA_MAX_TIME);
&nbsp;
// set the default socket timeout value
@ini_set("default_socket_timeout",AA_RECV_TIME);
&nbsp;
// output implicitly
@ob_implicit_flush(1);
&nbsp;
// for binary freads
@set_magic_quotes_runtime(0);
&nbsp;
// keep track of script execution time
$aa_time=time();
&nbsp;
// download each of these urls using fsockopen
aa_dl(&#039;http://httpd.apache.org&#039;);
aa_dl(&#039;http://www.w3.org&#039;);
aa_dl(&#039;http://www.google.com&#039;);
aa_dl(&#039;http://www.freebsd.org/cgi/man.cgi?query=connect&amp;sektion=2&amp;apropos=0&amp;manpath=FreeBSD+7.0-RELEASE&#039;);
aa_dl(&#039;http://www.askapache.com/htaccess/htaccess.html&#039;);
aa_dl(&#039;http://www.php.net&#039;);
aa_dl(&#039;http://en.wikipedia.org/wiki/Main_Page&#039;);
&nbsp;
/*  returns a socket pointer if valid or displays an error message
    sets stream timeout, starts the clock to check for socket read time */
function askapache_get_sock($target,$port){
  global $aa_time_start;
  $aa_time_start=time();
  if(false===($fp = @fsockopen($target,$port,$errno,$errstr,AA_CONN_TIME))||!is_resource($fp))
    return askapache_sock_strerror($errno,$errstr);
  @stream_set_timeout($fp, AA_RECV_TIME);
  return $fp;
}
&nbsp;
/*  writes request, then reads response until EOF, script max, or socket max
    returns response on success.  Uses buffer to allow size&gt;100megs */
function askapache_txrx($fp,$request,$chunk=1024){
  $rec=$buf=&#039;&#039;;
  if(!@fwrite($fp, $request, strlen($request)))die(&#039;fwrite error&#039;);
  while ( !@feof($fp) &amp;&amp; askapache_time_ok(askapache_time_passed())){
    $buf = @fread($fp, $chunk);
    $rec .= $buf;
  }
  if(!@fclose($fp))die(&#039;fclose error&#039;);
  return $rec;
}
&nbsp;
/* initiates the socket and download for the passed url.
   automatically handles gzip, chunked, both, and plain downloads.
   uses the long2ip/ip2long for ip validation, uses gethostbyname to
   get the ipv4 address which saves fsockopen from having to do the lookup
   final data is saved to $rbody but currently only displays headers.*/
function aa_dl($url=NULL){
  global $aa_time;
  $ub = @parse_url($url);
  if(!isset($ub[&#039;host&#039;])||empty($ub[&#039;host&#039;])) die("bad url $url");
  $proto   = ($ub[&#039;scheme&#039;]==&#039;https&#039;)?&#039;ssl://&#039;:&#039;&#039;;
  $port   = (isset($ub[&#039;port&#039;])&amp;&amp;!empty($ub[&#039;port&#039;])) ? $ub[&#039;port&#039;]:($proto!=&#039;&#039;)?443:80;
  $path   = (isset($ub[&#039;path&#039;])&amp;&amp;!empty($ub[&#039;path&#039;])) ? $ub[&#039;path&#039;]:&#039;/&#039;;
  $query   = (isset($ub[&#039;query&#039;])&amp;&amp;!empty($ub[&#039;query&#039;])) ? &#039;?&#039;.$ub[&#039;query&#039;] : &#039;&#039;;
  $host   = $ub[&#039;host&#039;];
  $ipp     = @gethostbyname($host);
  $ip     = ($ipp!=$host) ? long2ip(ip2long($ipp)) : $host;
&nbsp;
  $headers=array(
   "GET {$path}{$query} HTTP/1.1",
   "Host: {$host}",
   &#039;User-Agent: Mozilla/5.0 (AskApache/; +http://www.askapache.com/)&#039;,
   &#039;Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,*/*;q=0.5&#039;,
   &#039;Accept-Language: en-us,en;q=0.5&#039;,
   &#039;Accept-Encoding: gzip,deflate&#039;,
   &#039;Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7&#039;,
   &#039;Connection: close&#039;,&#039;Referer: http://www.askapache.com&#039;
  );
  $request=join(AA_LF,$headers).AA_LF.AA_LF;
&nbsp;
  $fp=askapache_get_sock($proto.$ip, $port);
  if($fp){
    $rbody=$rec=&#039;&#039;;$resp_headers=array();
    $rec=askapache_txrx($fp,$request);
    list($resp_headers, $rbody) = explode(AA_LF.AA_LF, trim($rec), 2);
  echo "\n&lt;p&gt;$request&lt;/p&gt;\n&lt;p&gt;$resp_headers&lt;/p&gt;\n";
    $gzip2=(stripos($resp_headers,&#039;Content-Encoding&#039;)!==false &amp;&amp;
        stripos($resp_headers,&#039;gzip&#039;)!==false)?1:0;
    $chunk=(stripos($resp_headers,&#039;Transfer-Encoding&#039;)!==false &amp;&amp;
        stripos($resp_headers,&#039;chunked&#039;)!==false)?1:0;
    $rbody=aa_decode_body($rbody,$chunk,$gzip2);
    unset($rbody);
  }
}
&nbsp;
/* based on http://us.php.net/manual/en/function.fsockopen.php#75175
   ungzips and/or re-assembles transfer-encoded:chunked responses
   returns the good response on success */
function aa_decode_body ($str, $chunked, $gzipped){
  if($gzipped &amp;&amp; !$chunked) return aa_gzdecode($str);
  if(!$gzipped &amp;&amp; !$chunked) return $str;
  $tmp = $str; $str = &#039;&#039;;
  do {
    $tmp = ltrim($tmp);
    $pos = strpos($tmp,AA_LF);
    $len = hexdec(substr($tmp, 0, $pos));
    if($gzipped) $str .= gzinflate(substr($tmp,($pos+12),$len));
    else $str .=substr($tmp,($pos+2),$len);
    $tmp = substr($tmp,($len+$pos+2));
  $chk=trim($tmp);
  } while (!empty($chk));
  return $str;
}
&nbsp;
/*  based on http://us2.php.net/manual/en/function.gzencode.php#82520
  saves the gzipped data to a tempfile, then outputs the decoded
  data to the output buffer using readgzfile, returning the decoded
  buffer and deleting the tempfile on success */
function aa_gzdecode($data){
  $g=tempnam(&#039;/tmp&#039;,&#039;ff&#039;);
  @file_put_contents($g,$data);
  ob_start(); readgzfile($g); $d=ob_get_clean(); @unlink($g);
  return $d;
}
&nbsp;
/*  very cool!  this is run during socket reads and checks whether the script
  execution time limit or the socket read time limit has been met, killing
  the script if so, otherwise returns true.  Run with a cron-like process */
function askapache_time_ok($sock_time=0) {
  global $aa_time;
  if (time()-$aa_time&gt;AA_MAX_TIME)
    die(&#039;killed script.. time exceeded &#039;.AA_MAX_TIME.&#039; Total: &#039;.$total);
  if ($sock_time&gt;AA_RECV_TIME)
    die(&#039;Killed socket.. time exceeded &#039;.AA_RECV_TIME.&#039; Total: &#039;.$sock_time);
  return true;
}
&nbsp;
/* input for askapache_time_ok to keep track of each socket read time time. */
function askapache_time_passed() {
  global $aa_time_start;
  return (time() - $aa_time_start);
}
&nbsp;
/*  handles fsockopen errors, printing them out though you may want to die on err */
function askapache_sock_strerror($errno,$errstr){
  switch($errno){
    case -3:  $err="Socket creation failed"; break;
    case -4:  $err="DNS lookup failure"; break;
    case -5:  $err="Connection refused or timed out"; break;
    case 111: $err="Connection refused"; break;
    case 113: $err="No route to host"; break;
    case 110: $err="Connection timed out"; break;
    case 104: $err="Connection reset by client"; break;
    default:  $err="Connection failed"; break;
  }
  echo &#039;&lt;p&gt;Fsockopen failed!&#039;."\n[".$errno."] ".$err." (".$errstr.")&lt;/p&gt;";
  return false;
}
?&gt;</pre>


<hr class="C" />




<h2>Debugging Fsockopen</h2>
<p>If you really want to know more about fsockopen, you can do what I did and read all the relevant php source files, your OS sys, lib, and user files relevant to fsockopen, and of course you can always trace php using the fsockopen function to get an under-the-hood look at what in the world fsockopen is doing.  Personally, I was trying to find more error codes and error strings to display when an fsockopen call failed, and I ended up finding over 50..</p>


<h3>fsockopen Errors</h3>
<pre>function fsockopen_err($errnum)
{
   static $fsockopen_errors;
   is_null($fsockopen_errors) &amp;&amp; $fsockopen_errors = array(
  0 =&gt; &#039;Success&#039;,
  1 =&gt; &#039;Operation not permitted&#039;,
  2 =&gt; &#039;No such file or directory&#039;,
  3 =&gt; &#039;No such process&#039;,
  4 =&gt; &#039;Interrupted system call - DNS lookup failure&#039;,
  5 =&gt; &#039;Input/output error - Connection refused or timed out&#039;,
  6 =&gt; &#039;No such device or address&#039;,
  7 =&gt; &#039;Argument list too long&#039;,
  8 =&gt; &#039;Exec format error&#039;,
  9 =&gt; &#039;Bad file descriptor&#039;,
  10 =&gt; &#039;No child processes&#039;,
  11 =&gt; &#039;Resource temporarily unavailable&#039;,
  12 =&gt; &#039;Cannot allocate memory&#039;,
  13 =&gt; &#039;Permission denied&#039;,
  14 =&gt; &#039;Bad address&#039;,
  15 =&gt; &#039;Block device required&#039;,
  16 =&gt; &#039;Device or resource busy&#039;,
  17 =&gt; &#039;File exists&#039;,
  18 =&gt; &#039;Invalid cross-device link&#039;,
  19 =&gt; &#039;No such device&#039;,
  20 =&gt; &#039;Not a directory&#039;,
  21 =&gt; &#039;Is a directory&#039;,
  22 =&gt; &#039;Invalid argument&#039;,
  23 =&gt; &#039;Too many open files in system&#039;,
  24 =&gt; &#039;Too many open files&#039;,
  25 =&gt; &#039;Inappropriate ioctl for device&#039;,
  26 =&gt; &#039;Text file busy&#039;,
  27 =&gt; &#039;File too large&#039;,
  28 =&gt; &#039;No space left on device&#039;,
  29 =&gt; &#039;Illegal seek&#039;,
  30 =&gt; &#039;Read-only file system&#039;,
  31 =&gt; &#039;Too many links&#039;,
  32 =&gt; &#039;Broken pipe&#039;,
  33 =&gt; &#039;Numerical argument out of domain&#039;,
  34 =&gt; &#039;Numerical result out of range&#039;,
  35 =&gt; &#039;Resource deadlock avoided&#039;,
  36 =&gt; &#039;File name too long&#039;,
  37 =&gt; &#039;No locks available&#039;,
  38 =&gt; &#039;Function not implemented&#039;,
  39 =&gt; &#039;Directory not empty&#039;,
  40 =&gt; &#039;Too many levels of symbolic links&#039;,
  41 =&gt; &#039;Unknown error 41&#039;,
  42 =&gt; &#039;No message of desired type&#039;,
  43 =&gt; &#039;Identifier removed&#039;,
  44 =&gt; &#039;Channel number out of range&#039;,
  45 =&gt; &#039;Level 2 not synchronized&#039;,
  46 =&gt; &#039;Level 3 halted&#039;,
  47 =&gt; &#039;Level 3 reset&#039;,
  48 =&gt; &#039;Link number out of range&#039;,
  49 =&gt; &#039;Protocol driver not attached&#039;,
  50 =&gt; &#039;No CSI structure available&#039;,
  51 =&gt; &#039;Level 2 halted&#039;,
  52 =&gt; &#039;Invalid exchange&#039;,
  53 =&gt; &#039;Invalid request descriptor&#039;,
  54 =&gt; &#039;Exchange full&#039;,
  55 =&gt; &#039;No anode&#039;,
  56 =&gt; &#039;Invalid request code&#039;,
  57 =&gt; &#039;Invalid slot&#039;,
  58 =&gt; &#039;Unknown error 58&#039;,
  59 =&gt; &#039;Bad font file format&#039;,
  60 =&gt; &#039;Device not a stream&#039;,
  61 =&gt; &#039;No data available&#039;,
  62 =&gt; &#039;Timer expired&#039;,
  63 =&gt; &#039;Out of streams resources&#039;,
  64 =&gt; &#039;Machine is not on the network&#039;,
  65 =&gt; &#039;Package not installed&#039;,
  66 =&gt; &#039;Object is remote&#039;,
  67 =&gt; &#039;Link has been severed&#039;,
  68 =&gt; &#039;Advertise error&#039;,
  69 =&gt; &#039;Srmount error&#039;,
  70 =&gt; &#039;Communication error on send&#039;,
  71 =&gt; &#039;Protocol error&#039;,
  72 =&gt; &#039;Multihop attempted&#039;,
  73 =&gt; &#039;RFS specific error&#039;,
  74 =&gt; &#039;Bad message&#039;,
  75 =&gt; &#039;Value too large for defined data type&#039;,
  76 =&gt; &#039;Name not unique on network&#039;,
  77 =&gt; &#039;File descriptor in bad state&#039;,
  78 =&gt; &#039;Remote address changed&#039;,
  79 =&gt; &#039;Can not access a needed shared library&#039;,
  80 =&gt; &#039;Accessing a corrupted shared library&#039;,
  81 =&gt; &#039;.lib section in a.out corrupted&#039;,
  82 =&gt; &#039;Attempting to link in too many shared libraries&#039;,
  83 =&gt; &#039;Cannot exec a shared library directly&#039;,
  84 =&gt; &#039;Invalid or incomplete multibyte or wide character&#039;,
  85 =&gt; &#039;Interrupted system call should be restarted&#039;,
  86 =&gt; &#039;Streams pipe error&#039;,
  87 =&gt; &#039;Too many users&#039;,
  88 =&gt; &#039;Socket operation on non-socket&#039;,
  89 =&gt; &#039;Destination address required&#039;,
  90 =&gt; &#039;Message too long&#039;,
  91 =&gt; &#039;Protocol wrong type for socket&#039;,
  92 =&gt; &#039;Protocol not available&#039;,
  93 =&gt; &#039;Protocol not supported&#039;,
  94 =&gt; &#039;Socket type not supported&#039;,
  95 =&gt; &#039;Operation not supported&#039;,
  96 =&gt; &#039;Protocol family not supported&#039;,
  97 =&gt; &#039;Address family not supported by protocol&#039;,
  98 =&gt; &#039;Address already in use&#039;,
  99 =&gt; &#039;Cannot assign requested address&#039;,
  100 =&gt; &#039;Network is down&#039;,
  101 =&gt; &#039;Network is unreachable&#039;,
  102 =&gt; &#039;Network dropped connection on reset&#039;,
  103 =&gt; &#039;Software caused connection abort&#039;,
  104 =&gt; &#039;Connection reset by peer&#039;,
  105 =&gt; &#039;No buffer space available&#039;,
  106 =&gt; &#039;Transport endpoint is already connected&#039;,
  107 =&gt; &#039;Transport endpoint is not connected&#039;,
  108 =&gt; &#039;Cannot send after transport endpoint shutdown&#039;,
  109 =&gt; &#039;Too many references: cannot splice&#039;,
  110 =&gt; &#039;Connection timed out&#039;,
  111 =&gt; &#039;Connection refused&#039;,
  112 =&gt; &#039;Host is down&#039;,
  113 =&gt; &#039;No route to host&#039;,
  114 =&gt; &#039;Operation already in progress&#039;,
  115 =&gt; &#039;Operation now in progress&#039;,
  116 =&gt; &#039;Stale NFS file handle&#039;,
  117 =&gt; &#039;Structure needs cleaning&#039;,
  118 =&gt; &#039;Not a XENIX named type file&#039;,
  119 =&gt; &#039;No XENIX semaphores available&#039;,
  120 =&gt; &#039;Is a named type file&#039;,
  121 =&gt; &#039;Remote I/O error&#039;,
  122 =&gt; &#039;Disk quota exceeded&#039;,
  123 =&gt; &#039;No medium found&#039;,
  124 =&gt; &#039;Wrong medium type&#039;,
  125 =&gt; &#039;Operation canceled&#039;
  );
    return (isset($fsockopen_errors[$errnum])) ? $fsockopen_errors[$errnum] : $errnum;
}</pre>
<p>If you would like to see all the errors on your particular machine:</p>
<pre>for($i=0, $s=""; $i&lt;250; $s=socket_strerror($i), $i++)
  !empty($s) &amp;&amp; (&#039;Unknown error&#039; != (substr($s,0,13)) ) &amp;&amp; print "{$i} =&gt; {$s}\n";</pre>
<p>Which outputs:</p>
<pre>1 =&gt; Success
2 =&gt; Operation not permitted
3 =&gt; No such file or directory
4 =&gt; No such process
5 =&gt; Interrupted system call
6 =&gt; Input/output error
7 =&gt; No such device or address
8 =&gt; Argument list too long
9 =&gt; Exec format error
10 =&gt; Bad file descriptor
11 =&gt; No child processes
12 =&gt; Resource temporarily unavailable
13 =&gt; Cannot allocate memory
14 =&gt; Permission denied
15 =&gt; Bad address
16 =&gt; Block device required
17 =&gt; Device or resource busy
18 =&gt; File exists
19 =&gt; Invalid cross-device link
20 =&gt; No such device
21 =&gt; Not a directory
22 =&gt; Is a directory
23 =&gt; Invalid argument
24 =&gt; Too many open files in system
25 =&gt; Too many open files
26 =&gt; Inappropriate ioctl for device
27 =&gt; Text file busy
28 =&gt; File too large
29 =&gt; No space left on device
30 =&gt; Illegal seek
31 =&gt; Read-only file system
32 =&gt; Too many links
33 =&gt; Broken pipe
34 =&gt; Numerical argument out of domain
35 =&gt; Numerical result out of range
36 =&gt; Resource deadlock avoided
37 =&gt; File name too long
38 =&gt; No locks available
39 =&gt; Function not implemented
40 =&gt; Directory not empty
41 =&gt; Too many levels of symbolic links
43 =&gt; No message of desired type
44 =&gt; Identifier removed
45 =&gt; Channel number out of range
46 =&gt; Level 2 not synchronized
47 =&gt; Level 3 halted
48 =&gt; Level 3 reset
49 =&gt; Link number out of range
50 =&gt; Protocol driver not attached
51 =&gt; No CSI structure available
52 =&gt; Level 2 halted
53 =&gt; Invalid exchange
54 =&gt; Invalid request descriptor
55 =&gt; Exchange full
56 =&gt; No anode
57 =&gt; Invalid request code
58 =&gt; Invalid slot
60 =&gt; Bad font file format
61 =&gt; Device not a stream
62 =&gt; No data available
63 =&gt; Timer expired
64 =&gt; Out of streams resources
65 =&gt; Machine is not on the network
66 =&gt; Package not installed
67 =&gt; Object is remote
68 =&gt; Link has been severed
69 =&gt; Advertise error
70 =&gt; Srmount error
71 =&gt; Communication error on send
72 =&gt; Protocol error
73 =&gt; Multihop attempted
74 =&gt; RFS specific error
75 =&gt; Bad message
76 =&gt; Value too large for defined data type
77 =&gt; Name not unique on network
78 =&gt; File descriptor in bad state
79 =&gt; Remote address changed
80 =&gt; Can not access a needed shared library
81 =&gt; Accessing a corrupted shared library
82 =&gt; .lib section in a.out corrupted
83 =&gt; Attempting to link in too many shared libraries
84 =&gt; Cannot exec a shared library directly
85 =&gt; Invalid or incomplete multibyte or wide character
86 =&gt; Interrupted system call should be restarted
87 =&gt; Streams pipe error
88 =&gt; Too many users
89 =&gt; Socket operation on non-socket
90 =&gt; Destination address required
91 =&gt; Message too long
92 =&gt; Protocol wrong type for socket
93 =&gt; Protocol not available
94 =&gt; Protocol not supported
95 =&gt; Socket type not supported
96 =&gt; Operation not supported
97 =&gt; Protocol family not supported
98 =&gt; Address family not supported by protocol
99 =&gt; Address already in use
100 =&gt; Cannot assign requested address
101 =&gt; Network is down
102 =&gt; Network is unreachable
103 =&gt; Network dropped connection on reset
104 =&gt; Software caused connection abort
105 =&gt; Connection reset by peer
106 =&gt; No buffer space available
107 =&gt; Transport endpoint is already connected
108 =&gt; Transport endpoint is not connected
109 =&gt; Cannot send after transport endpoint shutdown
110 =&gt; Too many references: cannot splice
111 =&gt; Connection timed out
112 =&gt; Connection refused
113 =&gt; Host is down
114 =&gt; No route to host
115 =&gt; Operation already in progress
116 =&gt; Operation now in progress
117 =&gt; Stale NFS file handle
118 =&gt; Structure needs cleaning
119 =&gt; Not a XENIX named type file
120 =&gt; No XENIX semaphores available
121 =&gt; Is a named type file
122 =&gt; Remote I/O error
123 =&gt; Disk quota exceeded
124 =&gt; No medium found
125 =&gt; Wrong medium type
126 =&gt; Operation canceled</pre>



<h3>Tracing fsockopen using Strace</h3>
<p>Once you save the above file on your site, you can use the strace tool to debug it.  This is a tad overboard but way cool nevertheless!</p>

<p><code>strace -e trace=connect php -nef fsockopen-test.php</code></p>
<pre>connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.33.216.129")}, 28) = 0
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("192.87.106.226")}, 16) = -1 EINPROGRESS (Operation now in progress)</pre>


<p><code>strace -e trace=network php -nef fsockopen-test.php</code></p>
<pre>socket(PF_FILE, SOCK_STREAM, 0)         = 3
connect(3, {sa_family=AF_FILE, path="/var/run/.nscd_socket"}, 110) = -1 ENOENT (No such file or directory)
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.33.216.129")}, 28) = 0
send(3, "\274\221\1\0\0\1\0\0\0\0\0\0\5httpd\6apache\3org\0\0\1"..., 34, 0) = 34
recvfrom(3, "\274\221\201\200\0\1\0\1\0\0\0\0\5httpd\6apache\3org\0"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.33.216.129")}, [16]) = 50
socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = -1 EAFNOSUPPORT (Address family not supported by protocol)
socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("192.87.106.226")}, 16) = -1 EINPROGRESS (Operation now in progress)
getsockopt(3, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
send(3, "GET / HTTP/1.1\r\nHost: httpd.apac"..., 356, MSG_DONTWAIT) = 356
recv(3, "HTTP/1.1 200 OK\r\nDate: Wed, 02 J"..., 8192, MSG_DONTWAIT) = 2609
recv(3, "", 8192, MSG_DONTWAIT)         = 0</pre>

<p><code>strace -q -e trace=all php -nef fsockopen-test.php</code></p>
<pre>mmap2(NULL, 266240, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb76ba000
munmap(0xb76ba000, 266240)              = 0
socket(PF_FILE, SOCK_STREAM, 0)         = 3
connect(3, {sa_family=AF_FILE, path="/var/run/.nscd_socket"}, 110) = -1 ENOENT (No such file or directory)
close(3)                                = 0
open("/etc/hosts", O_RDONLY)            = 3
fcntl64(3, F_GETFD)                     = 0
fcntl64(3, F_SETFD, FD_CLOEXEC)         = 0
fstat64(3, {st_mode=S_IFREG|0644, st_size=948, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f6e000
read(3, "# /etc/hosts - dh2 generated\n127"..., 4096) = 948
read(3, "", 4096)                       = 0
close(3)                                = 0
munmap(0xb7f6e000, 4096)                = 0
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.33.216.129")}, 28) = 0
send(3, "X~\1\0\0\1\0\0\0\0\0\0\2en\twikipedia\3org\0\0\1"..., 34, 0) = 34
gettimeofday({1214998196, 656179}, NULL) = 0
poll([{fd=3, events=POLLIN, revents=POLLIN}], 1, 5000) = 1
ioctl(3, FIONREAD, [100])               = 0
recvfrom(3, "X~\201\200\0\1\0\3\0\0\0\0\2en\twikipedia\3org\0\0\1"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.33.216.129")}, [16]) = 100
close(3)                                = 0
time(NULL)                              = 1214998196
gettimeofday({1214998196, 656754}, NULL) = 0
socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 3
fcntl64(3, F_GETFL)                     = 0x2 (flags O_RDWR)
fcntl64(3, F_SETFL, O_RDWR|O_NONBLOCK)  = 0
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("208.80.152.2")}, 16) = -1 EINPROGRESS (Operation now in progress)
poll([{fd=3, events=POLLIN|POLLOUT|POLLERR|POLLHUP, revents=POLLOUT}], 1, 10000) = 1
getsockopt(3, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
fcntl64(3, F_SETFL, O_RDWR)             = 0
send(3, "GET /wiki/Main_Page HTTP/1.1\r\nHo"..., 370, MSG_DONTWAIT) = 370
poll([{fd=3, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0
time(NULL)                              = 1214998196
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP, revents=POLLIN}], 1, 30000) = 1
recv(3, "HTTP/1.0 200 OK\r\nDate: Wed, 02 J"..., 8192, MSG_DONTWAIT) = 2896
time(NULL)                              = 1214998196
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP, revents=POLLIN}], 1, 30000) = 1
recv(3, "\214!\337i\307\336\23w\253wy\215\26EL\227;\227\253\261"..., 8192, MSG_DONTWAIT) = 5792
time(NULL)                              = 1214998196
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP, revents=POLLIN}], 1, 30000) = 1
recv(3, "4\201\273\214\17yI\347\257\371\373\344\332\330\227\245"..., 8192, MSG_DONTWAIT) = 7487
time(NULL)                              = 1214998197
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP, revents=POLLIN}], 1, 30000) = 1
recv(3, "", 8192, MSG_DONTWAIT)         = 0
close(3)                                = 0
write(1, "\n&lt;pre&gt;GET /wiki/Main_Page HTTP/1"..., 1300</pre>




<hr class="C" />


<h2>More Fsockopen Info</h2>

<h3>TCP Multiplexing</h3>
<p><a href="http://rfc.askapache.com/rfc793/rfc793.html#page-10">RFC 793</a>: To allow for many processes within a single Host to use TCP communication facilities simultaneously, the TCP provides a set of addresses or ports within each host.  Concatenated with the network and host addresses from the internet communication layer, this forms a socket.  A pair of sockets uniquely identifies each connection. That is, a socket may be simultaneously used in multiple connections.</p>
<p>The binding of ports to processes is handled independently by each Host.  However, it proves useful to attach frequently used processes (e.g., a "logger" or timesharing service) to fixed sockets which are made known to the public.  These services can then be accessed through the known addresses.  Establishing and learning the port addresses of other processes may involve more dynamic mechanisms.</p>

<h3>TCP Connections</h3>
<p>The reliability and flow control mechanisms described above require that TCPs initialize and maintain certain status information for each data stream.  The combination of this information, including sockets, sequence numbers, and window sizes, is called a connection. Each connection is uniquely specified by a pair of sockets identifying its two sides.</p>
<p>When two processes wish to communicate, their TCP's must first establish a connection (initialize the status information on each side).  When their communication is complete, the connection is terminated or closed to free the resources for other uses.</p>
<p>Since connections must be established between unreliable hosts and over the unreliable internet communication system, a handshake mechanism with clock-based sequence numbers is used to avoid erroneous initialization of connections.</p>

<h3>Fsockopen Practical Uses</h3>
<ul>
<li>Download Web Pages, Files, etc.</li>
<li><a href="http://www.askapache.com/php/speedy-form-post.html">Upload a file</a></li>
<li><a href="http://www.askapache.com/php/speedy-form-post.html">Send POST data to a form</a></li>
<li>Emulate cron</li>
<li>Download plugin updates</li>
<li><a href="http://www.askapache.com/online-tools/request-method-scanner/">Scan sites for exploits</a></li>
<li><a href="http://www.askapache.com/online-tools/curl-google-feed/">Auto Login to Google</a></li>
<li><a href="http://www.askapache.com/wordpress/crazy-cache-wordpress-plugin.html">Pass wp-nonces via cookie headers</a>, and more</li>
</ul>


<h3>Transfer-Encoding</h3>
<p><a href="http://www.ietf.org/rfc/rfc2068">RFC 2068</a></p>
<pre>19.4.6 Introduction of Transfer-Encoding
&nbsp;
   HTTP/1.1 introduces the Transfer-Encoding header field (section
   14.40).  Proxies/gateways MUST remove any transfer coding prior to
   forwarding a message via a MIME-compliant protocol.
&nbsp;
   A process for decoding the "chunked" transfer coding (section 3.6)
   can be represented in pseudo-code as:
&nbsp;
          length := 0
          read chunk-size, chunk-ext (if any) and CRLF
          while (chunk-size &gt; 0) {
             read chunk-data and CRLF
             append chunk-data to entity-body
             length := length + chunk-size
             read chunk-size and CRLF
          }
          read entity-header
          while (entity-header not empty) {
             append entity-header to existing header fields
             read entity-header
          }
          Content-Length := length
          Remove "chunked" from Transfer-Encoding</pre>


<h3>Socket-Related Man Pages</h3>
<pre>DESCRIPTION
This  manual  page  describes the Linux networking socket layer user interface. The BSD compatible sockets are the uniform interface between
the user process and the network protocol stacks in the kernel.  The protocol modules are  grouped  into  protocol  families  like  PF_INET,
PF_IPX, PF_PACKET and socket types like SOCK_STREAM or SOCK_DGRAM.  See socket(2) for more information on families and types.
&nbsp;
SOCKET LAYER FUNCTIONS
These  functions  are  used by the user process to send or receive packets and to do other socket operations. For more information see their
respective manual pages.
&nbsp;
socket(2) creates a socket, connect(2) connects a socket to a remote socket address, the bind(2) function binds a socket to a  local  socket
address,  listen(2)  tells  the socket that new connections shall be accepted, and accept(2) is used to get a new socket with a new incoming
connection.  socketpair(2) returns two connected anonymous sockets (only implemented for a few local families like PF_UNIX)
&nbsp;
send(2), sendto(2), and sendmsg(2) send data over a socket, and recv(2), recvfrom(2), recvmsg(2) receive data from a  socket.   poll(2)  and
select(2)  wait  for  arriving  data  or a readiness to send data.  In addition, the standard I/O operations like write(2), writev(2), send-
file(2), read(2), and readv(2) can be used to read and write data.
&nbsp;
getsockname(2) returns the local socket address and getpeername(2) returns the remote socket address.  getsockopt(2) and  setsockopt(2)  are
used to set or get socket layer or protocol options.  ioctl(2) can be used to set or read some other options.
&nbsp;
close(2) is used to close a socket.  shutdown(2) closes parts of a full duplex socket connection.
&nbsp;
Seeking, or calling pread(2) or pwrite(2) with a non-zero position is not supported on sockets.
&nbsp;
It  is possible to do non-blocking IO on sockets by setting the O_NONBLOCK flag on a socket file descriptor using fcntl(2).  Then all opera-
tions that would block will (usually) return with EAGAIN (operation should be retried later); connect(2) will return EINPROGRESS error.  The
user can then wait for various events via poll(2) or select(2).</pre>

<p>From the <a href="http://www.freebsd.org/cgi/man.cgi?query=socket&sektion=2&apropos=0&manpath=FreeBSD+7.0-RELEASE">FreeBSD man page for socket(2)</a></p>
<pre>Sockets of type SOCK_STREAM are full-duplex byte streams, similar to
pipes.  A stream socket must be in a connected state before any data may
be sent or received on it.  A connection to another socket is created
with a connect(2) system call.  Once connected, data may be transferred
using read(2) and write(2) calls or some variant of the send(2) and
recv(2) functions.  (Some protocol families, such as the Internet family,
support the notion of an ``implied connect&#039;&#039;, which permits data to be
sent piggybacked onto a connect operation by using the sendto(2) system
call.)  When a session has been completed a close(2) may be performed.
Out-of-band data may also be transmitted as described in send(2) and
received as described in recv(2).
&nbsp;
The communications protocols used to implement a SOCK_STREAM insure that
data is not lost or duplicated.  If a piece of data for which the peer
protocol has buffer space cannot be successfully transmitted within a
reasonable length of time, then the connection is considered broken and
calls will indicate an error with -1 returns and with ETIMEDOUT as the
specific code in the global variable errno.  The protocols optionally
keep sockets ``warm&#039;&#039; by forcing transmissions roughly every minute in
the absence of other activity.  An error is then indicated if no response
can be elicited on an otherwise idle connection for an extended period
(e.g. 5 minutes).  A SIGPIPE signal is raised if a process sends on a
broken stream; this causes naive processes, which do not handle the sig-
nal, to exit.</pre>

<p>Have Fun   ;)</p>




<pre>define (&#039;SOCKET_EPERM&#039;, 1);
define (&#039;SOCKET_ENOENT&#039;, 2);
define (&#039;SOCKET_EINTR&#039;, 4);
define (&#039;SOCKET_EIO&#039;, 5);
define (&#039;SOCKET_ENXIO&#039;, 6);
define (&#039;SOCKET_E2BIG&#039;, 7);
define (&#039;SOCKET_EBADF&#039;, 9);
define (&#039;SOCKET_EAGAIN&#039;, 11);
define (&#039;SOCKET_ENOMEM&#039;, 12);
define (&#039;SOCKET_EACCES&#039;, 13);
define (&#039;SOCKET_EFAULT&#039;, 14);
define (&#039;SOCKET_ENOTBLK&#039;, 15);
define (&#039;SOCKET_EBUSY&#039;, 16);
define (&#039;SOCKET_EEXIST&#039;, 17);
define (&#039;SOCKET_EXDEV&#039;, 18);
define (&#039;SOCKET_ENODEV&#039;, 19);
define (&#039;SOCKET_ENOTDIR&#039;, 20);
define (&#039;SOCKET_EISDIR&#039;, 21);
define (&#039;SOCKET_EINVAL&#039;, 22);
define (&#039;SOCKET_ENFILE&#039;, 23);
define (&#039;SOCKET_EMFILE&#039;, 24);
define (&#039;SOCKET_ENOTTY&#039;, 25);
define (&#039;SOCKET_ENOSPC&#039;, 28);
define (&#039;SOCKET_ESPIPE&#039;, 29);
define (&#039;SOCKET_EROFS&#039;, 30);
define (&#039;SOCKET_EMLINK&#039;, 31);
define (&#039;SOCKET_EPIPE&#039;, 32);
define (&#039;SOCKET_ENAMETOOLONG&#039;, 36);
define (&#039;SOCKET_ENOLCK&#039;, 37);
define (&#039;SOCKET_ENOSYS&#039;, 38);
define (&#039;SOCKET_ENOTEMPTY&#039;, 39);
define (&#039;SOCKET_ELOOP&#039;, 40);
define (&#039;SOCKET_EWOULDBLOCK&#039;, 11);
define (&#039;SOCKET_ENOMSG&#039;, 42);
define (&#039;SOCKET_EIDRM&#039;, 43);
define (&#039;SOCKET_ECHRNG&#039;, 44);
define (&#039;SOCKET_EL2NSYNC&#039;, 45);
define (&#039;SOCKET_EL3HLT&#039;, 46);
define (&#039;SOCKET_EL3RST&#039;, 47);
define (&#039;SOCKET_ELNRNG&#039;, 48);
define (&#039;SOCKET_EUNATCH&#039;, 49);
define (&#039;SOCKET_ENOCSI&#039;, 50);
define (&#039;SOCKET_EL2HLT&#039;, 51);
define (&#039;SOCKET_EBADE&#039;, 52);
define (&#039;SOCKET_EBADR&#039;, 53);
define (&#039;SOCKET_EXFULL&#039;, 54);
define (&#039;SOCKET_ENOANO&#039;, 55);
define (&#039;SOCKET_EBADRQC&#039;, 56);
define (&#039;SOCKET_EBADSLT&#039;, 57);
define (&#039;SOCKET_ENOSTR&#039;, 60);
define (&#039;SOCKET_ENODATA&#039;, 61);
define (&#039;SOCKET_ETIME&#039;, 62);
define (&#039;SOCKET_ENOSR&#039;, 63);
define (&#039;SOCKET_ENONET&#039;, 64);
define (&#039;SOCKET_EREMOTE&#039;, 66);
define (&#039;SOCKET_ENOLINK&#039;, 67);
define (&#039;SOCKET_EADV&#039;, 68);
define (&#039;SOCKET_ESRMNT&#039;, 69);
define (&#039;SOCKET_ECOMM&#039;, 70);
define (&#039;SOCKET_EPROTO&#039;, 71);
define (&#039;SOCKET_EMULTIHOP&#039;, 72);
define (&#039;SOCKET_EBADMSG&#039;, 74);
define (&#039;SOCKET_ENOTUNIQ&#039;, 76);
define (&#039;SOCKET_EBADFD&#039;, 77);
define (&#039;SOCKET_EREMCHG&#039;, 78);
define (&#039;SOCKET_ERESTART&#039;, 85);
define (&#039;SOCKET_ESTRPIPE&#039;, 86);
define (&#039;SOCKET_EUSERS&#039;, 87);
define (&#039;SOCKET_ENOTSOCK&#039;, 88);
define (&#039;SOCKET_EDESTADDRREQ&#039;, 89);
define (&#039;SOCKET_EMSGSIZE&#039;, 90);
define (&#039;SOCKET_EPROTOTYPE&#039;, 91);
define (&#039;SOCKET_ENOPROTOOPT&#039;, 92);
define (&#039;SOCKET_EPROTONOSUPPORT&#039;, 93);
define (&#039;SOCKET_ESOCKTNOSUPPORT&#039;, 94);
define (&#039;SOCKET_EOPNOTSUPP&#039;, 95);
define (&#039;SOCKET_EPFNOSUPPORT&#039;, 96);
define (&#039;SOCKET_EAFNOSUPPORT&#039;, 97);
define (&#039;SOCKET_EADDRINUSE&#039;, 98);
define (&#039;SOCKET_EADDRNOTAVAIL&#039;, 99);
define (&#039;SOCKET_ENETDOWN&#039;, 100);
define (&#039;SOCKET_ENETUNREACH&#039;, 101);
define (&#039;SOCKET_ENETRESET&#039;, 102);
define (&#039;SOCKET_ECONNABORTED&#039;, 103);
define (&#039;SOCKET_ECONNRESET&#039;, 104);
define (&#039;SOCKET_ENOBUFS&#039;, 105);
define (&#039;SOCKET_EISCONN&#039;, 106);
define (&#039;SOCKET_ENOTCONN&#039;, 107);
define (&#039;SOCKET_ESHUTDOWN&#039;, 108);
define (&#039;SOCKET_ETOOMANYREFS&#039;, 109);
define (&#039;SOCKET_ETIMEDOUT&#039;, 110);
define (&#039;SOCKET_ECONNREFUSED&#039;, 111);
define (&#039;SOCKET_EHOSTDOWN&#039;, 112);
define (&#039;SOCKET_EHOSTUNREACH&#039;, 113);
define (&#039;SOCKET_EALREADY&#039;, 114);
define (&#039;SOCKET_EINPROGRESS&#039;, 115);
define (&#039;SOCKET_EISNAM&#039;, 120);
define (&#039;SOCKET_EREMOTEIO&#039;, 121);
define (&#039;SOCKET_EDQUOT&#039;, 122);
define (&#039;SOCKET_ENOMEDIUM&#039;, 123);
define (&#039;SOCKET_EMEDIUMTYPE&#039;, 124);</pre>


<ul>
<li><a href="http://www.w3.org/Protocols/rfc2616/rfc2616.html">Hypertext Transfer Protocol — HTTP/1.1</a>, RFC 2616.  R. Fielding <em>et al.</em></li>
<li><a href="http://www.w3.org/Talks/9608HTTP/index.htm"> Hypertext ransport Protocol HTTP/1.1</a>.  J. Gettys. (slides)</li>
<li><a href="http://www.usenix.org/publications/library/proceedings/usenix99/invited_talks/mogul.pdf">What's wrong with HTTP (and why it doesn't matter)</a>.J. C. Mogul. (PDF slides)</li>
<li><a href="http://www.w3.org/Protocols/HTTP/Performance/Pipeline.html">Network Performance Effects of HTTP/1.1, CSS1, and PNG</a>.H. F. Nielsen, J. Gettys <em>et al.</em></li>
<li>Mozilla's <a href="http://www.mozilla.org/projects/netlib/http/pipelining-faq.html">HTTP/1.1 Pipelining FAQ</a>. D. Fisher.</li>
<li>Wikipedia: <a href="http://en.wikipedia.org/wiki/HTTP_proxy">HTTP proxy</a>.</li>
</ul><p><a href="http://www.askapache.com/php/fsockopen-socket.html"></a><a href="http://www.askapache.com/php/fsockopen-socket.html">Fsockopen Power Plays</a> originally appeared on <cite>AskApache.com</cite> </p>]]></content:encoded>
			<wfw:commentRss>http://www.askapache.com/php/fsockopen-socket.html/feed</wfw:commentRss>
		<slash:comments>4</slash:comments>
		</item>
	</channel>
</rss>

