AskApache Logo Fsockopen Power Plays

Search AskApache

1 diggs

Updated: Jul 02, 08 | Fsockopen Power Plays

« SEO Boost from Google 404 PluginEncrypted WordPress / phpBB Backups »

PHP’s function fsockopen lets you open an Internet or Unix domain socket connection for connecting to a resource, and is one of the most powerful functions. fsockopen could be described as creating a direct link to the wire connected to a resource, which means you can send any information (EBCDIC, ASCII, Hex, C arrays, Raw) directly to the target server.

A Socket is like /dev/null

In unix you can send anything to the /dev/null device, for Windows think Recycle Bin, and likewise you can send anything to a socket created with fsockopen. I’ve seen fsockopen code that sends custom exploits to cisco routers, including being used by the metasploit framework. I’ve seen fsockopen telnet emulation, smtp/pop3 login, and a lot of other advanced raw networking that is exciting for me see.

Some Definitions for Fsockopen

client
A program that establishes connections for the purpose of sending requests.
server
An application program that accepts connections in order to service requests by sending back responses.

Simple Socket Explantion

A web server host listens on TCP port 80. When a client host wishes to view a resource on the web server, it establishes a TCP connection with the server host by opening a socket to send the request for the resource. When the connection is established, the client and server exchange requests and responses (respectively) until the connection is closed or aborted.

HTTP and fsockopen

Snoopy Fsockopen HTTP Class for PHPThe Snoopy class is bundled with WordPress distributions and uses fsockopen to achieve most of its cool features. WordPress core, plugins, and other included files and classes also use the fsockopen function to communicate via HTTP.

Fsockopen Examples

fsockopen warningNote the warning sign, fsockopen is dangerous in the sense that you can crash your server, perform a DOS against your own server or other site, use up all your servers available sockets and fd descriptors, use up your bandwidth, etc.. Shouldn’t be a problem unless you are being malicious or careless.

Here are some BOSS fsockopen functions I hacked together yesterday for use in my AskApache Crazy Cache WordPress Plugin. I’ve used code and ideas from 100’s of authors, projects, and docs to try to make this the very best I can.

Intro

This is a working example employing as many of the best-practices, tips, and tricks for using fsockopen on remote streams that I could find.

<?php
// max time for script execution
if(!@defined('AA_MAX_TIME')) define('AA_MAX_TIME',  60);
 
// max time for socket reads
if(!@defined('AA_RECV_TIME')) define('AA_RECV_TIME', 30);
 
// max time for socket connect
if(!@defined('AA_CONN_TIME')) define('AA_CONN_TIME', 5);
 
// linebreak
if(!@defined('AA_LF')) define('AA_LF', chr(13).chr(10));
 
// ignore TCP RST i.e. browser stop button
@ignore_user_abort(1);
 
// set the script execution time
@set_time_limit(AA_MAX_TIME); 
 
// set the default socket timeout value
@ini_set("default_socket_timeout",AA_RECV_TIME);
 
// output implicitly
@ob_implicit_flush(1);
 
// for binary freads
@set_magic_quotes_runtime(0);
 
// keep track of script execution time
$aa_time=time();
 
// download each of these urls using fsockopen 
aa_dl('http://httpd.apache.org');
aa_dl('http://www.w3.org');
aa_dl('http://www.google.com');
aa_dl('http://www.freebsd.org/cgi/man.cgi?query=connect&sektion=2&apropos=0&manpath=FreeBSD+7.0-RELEASE');
aa_dl('http://www.askapache.com/htaccess/apache-htaccess.html');
aa_dl('http://www.php.net');
aa_dl('http://en.wikipedia.org/wiki/Main_Page');
 
/*  returns a socket pointer if valid or displays an error message
    sets stream timeout, starts the clock to check for socket read time */
function askapache_get_sock($target,$port){
  global $aa_time_start;
  $aa_time_start=time();
  if(false===($fp = @fsockopen($target,$port,$errno,$errstr,AA_CONN_TIME))||!is_resource($fp)) 
    return askapache_sock_strerror($errno,$errstr);
  @stream_set_timeout($fp, AA_RECV_TIME);
  return $fp;
}
 
/*  writes request, then reads response until EOF, script max, or socket max
    returns response on success.  Uses buffer to allow size>100megs */
function askapache_txrx($fp,$request,$chunk=1024){
  $rec=$buf='';
  if(!@fwrite($fp, $request, strlen($request)))die('fwrite error');
  while ( !@feof($fp) && askapache_time_ok(askapache_time_passed())){
    $buf = @fread($fp, $chunk);
    $rec .= $buf;
  }
  if(!@fclose($fp))die('fclose error');
  return $rec;
}
 
/* initiates the socket and download for the passed url.
   automatically handles gzip, chunked, both, and plain downloads.
   uses the long2ip/ip2long for ip validation, uses gethostbyname to 
   get the ipv4 address which saves fsockopen from having to do the lookup
   final data is saved to $rbody but currently only displays headers.*/
function aa_dl($url=NULL){
  global $aa_time;
  $ub = @parse_url($url);
  if(!isset($ub['host'])||empty($ub['host'])) die("bad url $url"); 
  $proto   = ($ub['scheme']=='https')?'ssl://':'';
  $port   = (isset($ub['port'])&&!empty($ub['port'])) ? $ub['port']:($proto!='')?443:80;
  $path   = (isset($ub['path'])&&!empty($ub['path'])) ? $ub['path']:'/';
  $query   = (isset($ub['query'])&&!empty($ub['query'])) ? '?'.$ub['query'] : '';
  $host   = $ub['host'];
  $ipp     = @gethostbyname($host);
  $ip     = ($ipp!=$host) ? long2ip(ip2long($ipp)) : $host;
  
  $headers=array(
   "GET {$path}{$query} HTTP/1.1",
   "Host: {$host}",
   'User-Agent: Mozilla/5.0 (AskApache/; +http://www.askapache.com/)',
   'Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,*/*;q=0.5',
   'Accept-Language: en-us,en;q=0.5',
   'Accept-Encoding: gzip,deflate',
   'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7',
   'Connection: close','Referer: http://www.askapache.com'
  );
  $request=join(AA_LF,$headers).AA_LF.AA_LF;
  
  $fp=askapache_get_sock($proto.$ip, $port);
  if($fp){
    $rbody=$rec='';$resp_headers=array();
    $rec=askapache_txrx($fp,$request);
    list($resp_headers, $rbody) = explode(AA_LF.AA_LF, trim($rec), 2);
  echo "\n<p>$request</p>\n<p>$resp_headers</p>\n";
    $gzip2=(stripos($resp_headers,'Content-Encoding')!==false && 
        stripos($resp_headers,'gzip')!==false)?1:0;
    $chunk=(stripos($resp_headers,'Transfer-Encoding')!==false && 
        stripos($resp_headers,'chunked')!==false)?1:0;
    $rbody=aa_decode_body($rbody,$chunk,$gzip2);
    unset($rbody);
  }
}
 
/* based on http://us.php.net/manual/en/function.fsockopen.php#75175 
   ungzips and/or re-assembles transfer-encoded:chunked responses
   returns the good response on success */
function aa_decode_body ($str, $chunked, $gzipped){
  if($gzipped && !$chunked) return aa_gzdecode($str);
  if(!$gzipped && !$chunked) return $str;
  $tmp = $str; $str = '';
  do {
    $tmp = ltrim($tmp);
    $pos = strpos($tmp,AA_LF);
    $len = hexdec(substr($tmp, 0, $pos));
    if($gzipped) $str .= gzinflate(substr($tmp,($pos+12),$len));
    else $str .=substr($tmp,($pos+2),$len);
    $tmp = substr($tmp,($len+$pos+2));
  $chk=trim($tmp);
  } while (!empty($chk));
  return $str;
}
 
/*  based on http://us2.php.net/manual/en/function.gzencode.php#82520 
  saves the gzipped data to a tempfile, then outputs the decoded
  data to the output buffer using readgzfile, returning the decoded
  buffer and deleting the tempfile on success */
function aa_gzdecode($data){
  $g=tempnam('/tmp','ff');
  @file_put_contents($g,$data);
  ob_start(); readgzfile($g); $d=ob_get_clean(); @unlink($g);
  return $d;
}
 
/*  very cool!  this is run during socket reads and checks whether the script
  execution time limit or the socket read time limit has been met, killing
  the script if so, otherwise returns true.  Run with a cron-like process */
function askapache_time_ok($sock_time=0) {
  global $aa_time;
  if (time()-$aa_time>AA_MAX_TIME) 
    die('killed script.. time exceeded '.AA_MAX_TIME.' Total: '.$total);
  if ($sock_time>AA_RECV_TIME) 
    die('Killed socket.. time exceeded '.AA_RECV_TIME.' Total: '.$sock_time);
  return true;
}
 
/* input for askapache_time_ok to keep track of each socket read time time. */
function askapache_time_passed() {
  global $aa_time_start;
  return (time() - $aa_time_start);
}
 
/*  handles fsockopen errors, printing them out though you may want to die on err */
function askapache_sock_strerror($errno,$errstr){
  switch($errno){
    case -3:  $err="Socket creation failed"; break;
    case -4:  $err="DNS lookup failure"; break;
    case -5:  $err="Connection refused or timed out"; break;
    case 111: $err="Connection refused"; break;
    case 113: $err="No route to host"; break;
    case 110: $err="Connection timed out"; break;
    case 104: $err="Connection reset by client"; break;
    default:  $err="Connection failed"; break;
  }
  echo '<p>Fsockopen failed!'."\n[".$errno."] ".$err." (".$errstr.")</p>";
  return false;
}
?>

Debugging Fsockopen

If you really want to know more about fsockopen, you can do what I did and read all the relevant php source files, your OS sys, lib, and user files relevant to fsockopen, and of course you can always trace php using the fsockopen function to get an under-the-hood look at what in the world fsockopen is doing. Personally, I was trying to find more error codes and error strings to display when an fsockopen call failed, and I ended up finding over 50..

Tracing fsockopen using Strace

Once you save the above file on your site, you can use the strace tool to debug it. This is a tad overboard but way cool nevertheless!

strace -e trace=connect php -nef fsockopen-test.php

connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.33.216.129")}, 28) = 0
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("192.87.106.226")}, 16) = -1 EINPROGRESS (Operation now in progress)

strace -e trace=network php -nef fsockopen-test.php

socket(PF_FILE, SOCK_STREAM, 0)         = 3
connect(3, {sa_family=AF_FILE, path="/var/run/.nscd_socket"}, 110) = -1 ENOENT (No such file or directory)
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.33.216.129")}, 28) = 0
send(3, "\274\221\1\0\0\1\0\0\0\0\0\0\5httpd\6apache\3org\0\0\1"..., 34, 0) = 34
recvfrom(3, "\274\221\201\200\0\1\0\1\0\0\0\0\5httpd\6apache\3org\0"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.33.216.129")}, [16]) = 50
socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = -1 EAFNOSUPPORT (Address family not supported by protocol)
socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("192.87.106.226")}, 16) = -1 EINPROGRESS (Operation now in progress)
getsockopt(3, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
send(3, "GET / HTTP/1.1\r\nHost: httpd.apac"..., 356, MSG_DONTWAIT) = 356
recv(3, "HTTP/1.1 200 OK\r\nDate: Wed, 02 J"..., 8192, MSG_DONTWAIT) = 2609
recv(3, "", 8192, MSG_DONTWAIT)         = 0

strace -q -e trace=all php -nef fsockopen-test.php

mmap2(NULL, 266240, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb76ba000
munmap(0xb76ba000, 266240)              = 0
socket(PF_FILE, SOCK_STREAM, 0)         = 3
connect(3, {sa_family=AF_FILE, path="/var/run/.nscd_socket"}, 110) = -1 ENOENT (No such file or directory)
close(3)                                = 0
open("/etc/hosts", O_RDONLY)            = 3
fcntl64(3, F_GETFD)                     = 0
fcntl64(3, F_SETFD, FD_CLOEXEC)         = 0
fstat64(3, {st_mode=S_IFREG|0644, st_size=948, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f6e000
read(3, "# /etc/hosts - dh2 generated\n127"..., 4096) = 948
read(3, "", 4096)                       = 0
close(3)                                = 0
munmap(0xb7f6e000, 4096)                = 0
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.33.216.129")}, 28) = 0
send(3, "X~\1\0\0\1\0\0\0\0\0\0\2en\twikipedia\3org\0\0\1"..., 34, 0) = 34
gettimeofday({1214998196, 656179}, NULL) = 0
poll([{fd=3, events=POLLIN, revents=POLLIN}], 1, 5000) = 1
ioctl(3, FIONREAD, [100])               = 0
recvfrom(3, "X~\201\200\0\1\0\3\0\0\0\0\2en\twikipedia\3org\0\0\1"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("66.33.216.129")}, [16]) = 100
close(3)                                = 0
time(NULL)                              = 1214998196
gettimeofday({1214998196, 656754}, NULL) = 0
socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 3
fcntl64(3, F_GETFL)                     = 0x2 (flags O_RDWR)
fcntl64(3, F_SETFL, O_RDWR|O_NONBLOCK)  = 0
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("208.80.152.2")}, 16) = -1 EINPROGRESS (Operation now in progress)
poll([{fd=3, events=POLLIN|POLLOUT|POLLERR|POLLHUP, revents=POLLOUT}], 1, 10000) = 1
getsockopt(3, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
fcntl64(3, F_SETFL, O_RDWR)             = 0
send(3, "GET /wiki/Main_Page HTTP/1.1\r\nHo"..., 370, MSG_DONTWAIT) = 370
poll([{fd=3, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 1, 0) = 0
time(NULL)                              = 1214998196
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP, revents=POLLIN}], 1, 30000) = 1
recv(3, "HTTP/1.0 200 OK\r\nDate: Wed, 02 J"..., 8192, MSG_DONTWAIT) = 2896
time(NULL)                              = 1214998196
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP, revents=POLLIN}], 1, 30000) = 1
recv(3, "\214!\337i\307\336\23w\253wy\215\26EL\227;\227\253\261"..., 8192, MSG_DONTWAIT) = 5792
time(NULL)                              = 1214998196
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP, revents=POLLIN}], 1, 30000) = 1
recv(3, "4\201\273\214\17yI\347\257\371\373\344\332\330\227\245"..., 8192, MSG_DONTWAIT) = 7487
time(NULL)                              = 1214998197
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP, revents=POLLIN}], 1, 30000) = 1
recv(3, "", 8192, MSG_DONTWAIT)         = 0
close(3)                                = 0
write(1, "\n<pre>GET /wiki/Main_Page HTTP/1"..., 1300

More Fsockopen Info

TCP Multiplexing

RFC 793: To allow for many processes within a single Host to use TCP communication facilities simultaneously, the TCP provides a set of addresses or ports within each host. Concatenated with the network and host addresses from the internet communication layer, this forms a socket. A pair of sockets uniquely identifies each connection. That is, a socket may be simultaneously used in multiple connections.

The binding of ports to processes is handled independently by each Host. However, it proves useful to attach frequently used processes (e.g., a “logger” or timesharing service) to fixed sockets which are made known to the public. These services can then be accessed through the known addresses. Establishing and learning the port addresses of other processes may involve more dynamic mechanisms.

TCP Connections

The reliability and flow control mechanisms described above require that TCPs initialize and maintain certain status information for each data stream. The combination of this information, including sockets, sequence numbers, and window sizes, is called a connection. Each connection is uniquely specified by a pair of sockets identifying its two sides.

When two processes wish to communicate, their TCP’s must first establish a connection (initialize the status information on each side). When their communication is complete, the connection is terminated or closed to free the resources for other uses.

Since connections must be established between unreliable hosts and over the unreliable internet communication system, a handshake mechanism with clock-based sequence numbers is used to avoid erroneous initialization of connections.

Fsockopen Practical Uses

Transfer-Encoding

RFC 2068

19.4.6 Introduction of Transfer-Encoding
 
   HTTP/1.1 introduces the Transfer-Encoding header field (section
   14.40).  Proxies/gateways MUST remove any transfer coding prior to
   forwarding a message via a MIME-compliant protocol.
 
   A process for decoding the "chunked" transfer coding (section 3.6)
   can be represented in pseudo-code as:
 
          length := 0
          read chunk-size, chunk-ext (if any) and CRLF
          while (chunk-size > 0) {
             read chunk-data and CRLF
             append chunk-data to entity-body
             length := length + chunk-size
             read chunk-size and CRLF
          }
          read entity-header
          while (entity-header not empty) {
             append entity-header to existing header fields
             read entity-header
          }
          Content-Length := length
          Remove "chunked" from Transfer-Encoding

Socket-Related Man Pages

DESCRIPTION
This  manual  page  describes the Linux networking socket layer user interface. The BSD compatible sockets are the uniform interface between
the user process and the network protocol stacks in the kernel.  The protocol modules are  grouped  into  protocol  families  like  PF_INET,
PF_IPX, PF_PACKET and socket types like SOCK_STREAM or SOCK_DGRAM.  See socket(2) for more information on families and types.
 
SOCKET LAYER FUNCTIONS
These  functions  are  used by the user process to send or receive packets and to do other socket operations. For more information see their
respective manual pages.
 
socket(2) creates a socket, connect(2) connects a socket to a remote socket address, the bind(2) function binds a socket to a  local  socket
address,  listen(2)  tells  the socket that new connections shall be accepted, and accept(2) is used to get a new socket with a new incoming
connection.  socketpair(2) returns two connected anonymous sockets (only implemented for a few local families like PF_UNIX)
 
send(2), sendto(2), and sendmsg(2) send data over a socket, and recv(2), recvfrom(2), recvmsg(2) receive data from a  socket.   poll(2)  and
select(2)  wait  for  arriving  data  or a readiness to send data.  In addition, the standard I/O operations like write(2), writev(2), send-
file(2), read(2), and readv(2) can be used to read and write data.
 
getsockname(2) returns the local socket address and getpeername(2) returns the remote socket address.  getsockopt(2) and  setsockopt(2)  are
used to set or get socket layer or protocol options.  ioctl(2) can be used to set or read some other options.
 
close(2) is used to close a socket.  shutdown(2) closes parts of a full duplex socket connection.
 
Seeking, or calling pread(2) or pwrite(2) with a non-zero position is not supported on sockets.
 
It  is possible to do non-blocking IO on sockets by setting the O_NONBLOCK flag on a socket file descriptor using fcntl(2).  Then all opera-
tions that would block will (usually) return with EAGAIN (operation should be retried later); connect(2) will return EINPROGRESS error.  The
user can then wait for various events via poll(2) or select(2).

From the FreeBSD man page for socket(2)

Sockets of type SOCK_STREAM are full-duplex byte streams, similar to
pipes.  A stream socket must be in a connected state before any data may
be sent or received on it.  A connection to another socket is created
with a connect(2) system call.  Once connected, data may be transferred
using read(2) and write(2) calls or some variant of the send(2) and
recv(2) functions.  (Some protocol families, such as the Internet family,
support the notion of an ``implied connect'', which permits data to be
sent piggybacked onto a connect operation by using the sendto(2) system
call.)  When a session has been completed a close(2) may be performed.
Out-of-band data may also be transmitted as described in send(2) and
received as described in recv(2).
 
The communications protocols used to implement a SOCK_STREAM insure that
data is not lost or duplicated.  If a piece of data for which the peer
protocol has buffer space cannot be successfully transmitted within a
reasonable length of time, then the connection is considered broken and
calls will indicate an error with -1 returns and with ETIMEDOUT as the
specific code in the global variable errno.  The protocols optionally
keep sockets ``warm'' by forcing transmissions roughly every minute in
the absence of other activity.  An error is then indicated if no response
can be elicited on an otherwise idle connection for an extended period
(e.g. 5 minutes).  A SIGPIPE signal is raised if a process sends on a
broken stream; this causes naive processes, which do not handle the sig-
nal, to exit.

Have Fun ;)

Reader Comments

Skip to form

  1. MarcelloAugust 21, 2008 @ 10:30 am

    Hi, thanks for your example.
    I’ve run it and found a little bug, change line 75:

    $port   = (isset($ub['port']) && !empty($ub['port'])) ? $ub['port']:($proto!='')?443:80;
#to this
$port   = (isset($ub['port']) && !empty($ub['port'])) ? $ub['port']:(($proto!='')?443:80);

    I’ve added parenthesis around the second ‘?:’ operator otherwise it will takes precedence on the first and you’ll get your port ignored :(

    Ciao,
    – bm

  2. AskApacheJuly 3, 2008 @ 11:46 pm

    @Mike

    Good suggestion, I changed it to Windows… although I have many reasons for my feelings towards “the empire” that I won’t get into. I also fixed the print issue for you, thanks for your help dude!

  3. Mike SchinkelJuly 2, 2008 @ 2:21 pm

    GREAT, AWESOME STUFF!

    Unfortunately, your print stylesheet doesn’t work on this post, especially not using IE7 but also not using FF2.

    Also, can I suggest you loose the “windoze” references in the future? Your knowledge and skills are too significant that you really shouldn’t diminished yourself by self-selecting into the Microsoft-haters faction. The people with the resources to pay you the $175/hour you mention in your “Contact Me” form are usually the type that would view that such hating as immature.

Comment on "Fsockopen Power Plays"

I review each comment that makes it past my crazy-tight .htaccess anti-spam..
Please wrap code/source with <pre>...</pre> tags, (x)html is allowed and encouraged!

« SEO Boost from Google 404 PluginEncrypted WordPress / phpBB Backups »

Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution 3.0 License, which lets you use/modify/re-post this content provided you follow the attribution guidelines in the license.