SSLStream broken due to heap fragmentation  
Author Message
pkluches





PostPosted: Thu Nov 09 23:10:47 CST 2006 Top

Net Framework >> SSLStream broken due to heap fragmentation

We've been using the SSLStream class found in System.Net.Security to build a
giant Sockets server that provides TLS encryption at the channel leve.
Before .Net 2.0, we used an open-source encryption channel from Mentalis,
and have even looked at the Mono implementation for doing this.

The problem comes from the SSLStream not doing any buffer management. None.
Zero. In the "no buffer management" case, each SSLStream allocates bufferes
whenevers it needs them, copies data into those buffers, and then calls
Socket.BeingRead / Socket.BeginWrite. This causes pinning. Massive,
long-lived, horrible pins. This leads to horrendus non-recoverable
fragmentation.

The killer here is that both of these socket methods immediatly pin the
buffers, and pass the data off to unmanaged code. The BeginRead method may
not return for 10 seconds, 10 minutes, or 10 hours, leaving a hole in the
Managed Heap the entire time. BeginWrite causes the same problem, although
for a smaller length of time.

To get around this in the past, we've created a block of buffers ahead of
time, and cycled through them. This keeps all the pinned memory together in
a single spot. This technique is described in detail here:
http://www.hide-link.com/
http://www.hide-link.com/ ;EntryID=9

In our last implementation (.Net 1.1), we put alot of work into managing
these buffer pools and really trying hard to eliminate heap fragmentation.
This worked great.


With the .Net 2.0 SSLStream, the BeginRead method ends up allocating
bufferes here:
BeginRead->ProcessRead->EnsureInternalBufferSize
That method allocates a buffer by:
this._InternalBuffer = new byte[addSize + curOffset];

Notice there's no fregging pool here? No attempt to eliminate fragmentation.
No attempt to be efficient.

This buffer gets passed to the NetworkStream, and in turn to the Socket,
which promptly pins it.

The BeginWrite case is the same thing:
BeginWrite->ProcessWrite->StartWriting->EncryptBuffers->EncryptData->Encrypt
This method allocates it's buffer:
buffer1 = new byte[(size + this.m_HeaderSize) + this.m_TrailerSize];

Again, no fregging attempt to use a buffer pool. No attempt to eliminate
heap fragmentation.

Because the SSLStream is retarded, it only works on NetworkStreams - and
Network Streams only work on Sockets. This means there's nowhere in the
whole chain I can insert code prior to the Pin that would use a pooled
buffer. Ugh.

Now, .Net 2.0 is improved at managing pinning in the heap:
http://www.hide-link.com/

... but I have it on VERY good authority that their technique isn't nearly
as effective as the BufferPool method. In fact, it's not even close.

This has runied my whole day. Ugh.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise

DotNet106  
 
 
Chris





PostPosted: Thu Nov 09 23:10:47 CST 2006 Top

Net Framework >> SSLStream broken due to heap fragmentation We did figure out a way around this, in case anyone's interested.

By deriving a class from NetworkStream and overring the relevant methods, we
can swap out the buffers used by SSLStream with our own buffers. Because our
buffers come from a bufferpool that's allocated in a way to minimize heap
fragmentation, the heap stays unfragmented, and scalability is largely
unaffected.

We certainly end up doing number of buffer copies, but that's a much cheaper
price to pay than heap fragmentation.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise
http://www.coversant.net/blogs/cmullins


> We've been using the SSLStream class found in System.Net.Security to build
> a giant Sockets server that provides TLS encryption at the channel leve.
> Before .Net 2.0, we used an open-source encryption channel from Mentalis,
> and have even looked at the Mono implementation for doing this.
>
> The problem comes from the SSLStream not doing any buffer management.
> None. Zero. In the "no buffer management" case, each SSLStream allocates
> bufferes whenevers it needs them, copies data into those buffers, and then
> calls Socket.BeingRead / Socket.BeginWrite. This causes pinning. Massive,
> long-lived, horrible pins. This leads to horrendus non-recoverable
> fragmentation.
>
> The killer here is that both of these socket methods immediatly pin the
> buffers, and pass the data off to unmanaged code. The BeginRead method may
> not return for 10 seconds, 10 minutes, or 10 hours, leaving a hole in the
> Managed Heap the entire time. BeginWrite causes the same problem, although
> for a smaller length of time.
>
> To get around this in the past, we've created a block of buffers ahead of
> time, and cycled through them. This keeps all the pinned memory together
> in a single spot. This technique is described in detail here:
> http://blogs.msdn.com/yunjin/archive/2004/01/27/63642.aspx
> http://www.coversant.net/dotnetnuke/Default.aspx?tabid=88&EntryID=9
>
> In our last implementation (.Net 1.1), we put alot of work into managing
> these buffer pools and really trying hard to eliminate heap fragmentation.
> This worked great.
>
>
> With the .Net 2.0 SSLStream, the BeginRead method ends up allocating
> bufferes here:
> BeginRead->ProcessRead->EnsureInternalBufferSize
> That method allocates a buffer by:
> this._InternalBuffer = new byte[addSize + curOffset];
>
> Notice there's no fregging pool here? No attempt to eliminate
> fragmentation. No attempt to be efficient.
>
> This buffer gets passed to the NetworkStream, and in turn to the Socket,
> which promptly pins it.
>
> The BeginWrite case is the same thing:
> BeginWrite->ProcessWrite->StartWriting->EncryptBuffers->EncryptData->Encrypt
> This method allocates it's buffer:
> buffer1 = new byte[(size + this.m_HeaderSize) + this.m_TrailerSize];
>
> Again, no fregging attempt to use a buffer pool. No attempt to eliminate
> heap fragmentation.
>
> Because the SSLStream is retarded, it only works on NetworkStreams - and
> Network Streams only work on Sockets. This means there's nowhere in the
> whole chain I can insert code prior to the Pin that would use a pooled
> buffer. Ugh.
>
> Now, .Net 2.0 is improved at managing pinning in the heap:
> http://blogs.msdn.com/maoni/archive/2005/10/03/so-what-s-new-in-the-clr-2-0-gc.aspx
>
> ... but I have it on VERY good authority that their technique isn't nearly
> as effective as the BufferPool method. In fact, it's not even close.
>
> This has runied my whole day. Ugh.
>
> --
> Chris Mullins, MCSD.NET, MCPD:Enterprise
>