[vuln.sg] yEnc32 Decoder Long Filename Buffer Overflow Vulnerability

A vulnerability has been found in yEnc32. When exploited, the vulnerability allows execution of arbitrary code when the user decodes a specially crafted yEnc encoded file.

A heap-based buffer overflow vulnerability exists within yEnc32. The boundary error occurs when decoding a yEnc encoded file that contains an overly long filename. This can be exploited to corrupt the application heap and to overwrite function pointers on the heap.

Execution of arbitrary code using this vulnerability has been confirmed on English WinXP SP2 by making assumptions of the heap-buffer location, and overwriting a function pointer on the heap with the address of a memory location that contains a pointer to the heap-buffer. This approach is not 100% reliable, but nonetheless, proves that code execution is possible.

In order to exploit this vulnerability successfully, the user must be convinced to decode a malicious yEnc encoded file.

The following screen capture shows that it is possible to control the EIP via the overwritten function pointer on the heap.

The following POC NTX files will exploit the vulnerability to run calc.exe or crash yEnc32. The code execution POC has been successfully tested on English WinXP SP2.

Instructions to reproduce the vulnerability:

Download the POC files and save it to the hard-disk.
Run yEnc32.
Using yEnc32, select one of the POC file to decode.
In the "Save files to folder" box, type in "C:\". (Note: This exploit assumes that you are extracting to "C:\". Extracting to a different directory may cause the exploit to fail)

Click on the "OK" button to begin decoding.
Wait for the "Could not create output file: c:\AAAABBBB..." error dialog box to popup.
Close the error dialog box and try to click yEnc32's "File" menu, or try to close the yEnc32 window.
Successful exploit will run calc.exe or crash yEnc32.

2007-05-06 - Vulnerability discovered.
2007-05-09 - Initial vendor notification. (Email to support@, components@, sourcecode@ bounced)
2007-05-09 - Initial vendor notification. (Email to Sourceforge yEnc32 project admin)
2007-05-10 - Initial vendor reply.
2007-05-10 - Vulnerability description and POC files sent to vendor.
2007-05-12 - Received email from vendor that fixed version was released.
2007-05-12 - Public disclosure.