Monday, August 24, 2009

How to check signature in MySpace iframe application (.net version)

It was a kind of headache for me after I decided to add checking signature to our MySpace application. I couldn't make it work using existing tools like the MySpaceToolkit. But eventually I was able to do it myself. I cannot say it was painless. There were several examples in php like this one, that could bring even more questions than answers. But they meant to be working and my code started working too though having eaten a plenty of my time. Here is the complete code:

private static string GenerateIFrameSignature(HttpRequest req, string secretKey)
{
    const string c_unreservedChars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-_.~";

    // We cannot use the HttpUtility.UrlEncode method because it produces lowercase output as OAuth requires uppercase.
    // The code of the delegate was extracted out of MySpaceToolkit library.
    Func UrlEncode = (value) =>
    {
        StringBuilder result = new StringBuilder();

        foreach (char symbol in value)
        {
            if (c_unreservedChars.IndexOf(symbol) != -1)
            {
                result.Append(symbol);
            }
            else
            {
                result.Append('%' + String.Format("{0:X2}", (int)symbol));
            }
        }

        return result.ToString();
    };

    var sortedKeysWOSig = req.QueryString.AllKeys.Where(k => k != "oauth_signature").OrderBy(k => k);
    var query = new StringBuilder();

    foreach (string key in sortedKeysWOSig)
    {
        if (query.Length > 0) query.Append('&');

        query.Append(UrlEncode(key))
        .Append('=')
        .Append(UrlEncode(req.QueryString[key]));
    }

    StringBuilder @base = new StringBuilder();
        @base.Append("GET&")
        .Append(UrlEncode(req.Url.GetComponents(UriComponents.SchemeAndServer | UriComponents.Path, UriFormat.Unescaped)))
        .Append('&')
        .Append(UrlEncode(query.ToString()));


    using (KeyedHashAlgorithm alg = HMACSHA1.Create())
    {
        //according to http://oauth.googlecode.com/svn/spec/core/1.0/oauth-core-1_0.html#rfc.section.9.2 we should concat [Consumer Secret] + & + [Token Secrect]. The latter one is not provided by MySpace. So we imply empty string in this place.
        alg.Key = Encoding.ASCII.GetBytes(secretKey + "&");

        byte[] dataBuffer = Encoding.ASCII.GetBytes(@base.ToString());
        byte[] hashBytes = alg.ComputeHash(dataBuffer);

        return Convert.ToBase64String(hashBytes);
    }
}

There is no reason to use the UrlEncode as delegate except placing it in a single method for readability of this post.

After comparing signature out of 'oauth_signature' query string parameter and the one the method above returns you can also verify auth timestamp to be up to date:

private const string _oauthTimestampKey = "oauth_timestamp";
private static readonly TimeSpan _staledInterval = TimeSpan.FromMinutes(10);
private static readonly DateTime _stDate = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc);

...

long authTime;
if (!Int64.TryParse(request.QueryString[_oauthTimestampKey], NumberStyles.Any, CultureInfo.InvariantCulture, out authTime))
{
    #region Logging
    if (_log.IsDebugEnabled) _log.DebugFormat("Couldn't parse oauth_timestamp value {0}", authTime);
    #endregion
    return false;
}

var now = DateTime.UtcNow;
DateTime time = _stDate.AddSeconds(authTime);
if (now - time > _staledInterval)
{
    #region Logging
    if (_log.IsDebugEnabled) _log.DebugFormat("Auth time was too late. Expected greater than {0}. Actual was {1}", now - _staledInterval, time);
    #endregion
    return false;
}

Wednesday, June 10, 2009

How to run a shell command in the background on windows

If you are dealing wih unix os you can run command in background merely adding & in shell:

sleep 2 &

Recently I did need this functionality as I was debugging a web application crashes on our live servers. Being in terminal session I run the adplus script to catch crashes for a specified process:

adplus -crash -p <pid>

The script attaches debugger to a process making the debugger monitor exceptions. It could take a long while till we catch anything. I needed to leave my terminal session unattended or forcibly close it to proceed my regular routines. An unattended or disconnected session can be logged out according to a policy set by administrator, but the running script stuck to my session.

Fortunately, windows users have the magic 'at' command that schedules commands at a specified time. So we can run our debugging script in the background just by passing it the scheduler:

at <time> cscript "<fullpath-to>\adplus.vbs" -quiet -crash -p <pid>

We need the -quiet option to skip initial dialog box the script shows when attaching a process. We use absolute path to the adplus because the current directory for the executing command is the systemroot folder. Our command will be executed under System user. It's enough for debugging. But be careful when you rely on this, e.g. when you connect to sql server using integrated security.

I set the time parameter a minute ahead and the task is appointed on today.

Friday, March 27, 2009

DevSmtp server released

Have you ever debugged sending emails from your application? I have many times. Users of your website pay for goods and then they should be sent a notification. The conditions to test receiving mails can be uneasy. User account email should be valid and you are allowed to connect to smtp server or have your local smtp that might be considered by the remote one as unreliable and your mail will be rejected or cut off by a spam filter.

As a developer I always wanted a reliable tool for checking the both things: to ensure the email is sent and see how the email looks like for user.

Besides Windows Vista doesn't have an smtp server. So I decided to implement it on my own. It wasn't a real world project and I didn't have tight schedule, so I chose to refresh my C++ skills that had been more academical than practical and to write a server using I/O Completion Ports.

I use github to host the project. There you can either download the project source code and compile it in VS 2008 or get a precompiled binary.
  1. In order to run in console mode put devsmtp.exe in an appropriate location and execute: devsmtp -c
  2. to register as windows service, execute: devsmtp -i (to uninstall, use devsmtp -u)
Now start testing you programs. If you want to verify a message is sent, run DebugView utility and you will see smtp communication protocol messages between your application and the server. After the message is on the server, it will be stored in a folder under devsmtp.exe location named with from email.

So you can double click on the eml file and see it in embedded viewer.

Attention. The server doesn't send mails to recipients. It serves for the debugging purpose only.

Have fun!

Tuesday, January 27, 2009

How to fix Windows SDK Configuration Tool

There are many posts out there pointing out that Windows SDK Configuration Tool contains a bug. When I first met the issue I googled hither and thither but did find a solution for Visual Studio 2005. The workarounds did not work for me and were too clumsy to use them on a regular basis.

Eventually I decided to dig into its code - fortunately WindowsSdkVer.exe is a .net assembly. Therefore, bingo, we can use .net reflector to infer its sorce code. It wasn't too difficult. To that end, the reflector contains an option to export an entire assembly as the visual studio project:

 Export an assembly

The exported WindowsSdkVer project contained a dependency, the library VCIntegrate.dll, that can be found in the same folder as WindowsSdkVer.exe itself lies. I added the library to the project references and could build and run it.

Running the project on my Vista x64 I quickly found my issue. It was in the Utility.cs file, the GetInstalledProduct method:

foreach (string str in key.GetSubKeyNames())
{
    RegistryKey key2 = key.OpenSubKey(str);
    Product newProduct = new Product();
    try
    {
        newProduct.SdkVersion = str;
        if (!string.IsNullOrEmpty(productVersionKey))
        {
            newProduct.Version = new Version(key2.GetValue(productVersionKey).ToString());
            newProduct.Version = new Version(key2.GetValue(productVersionKey).ToString().Replace("v", "").Replace("A", ""));
        }
        else
        {

The problem is that the Version class can't parse version given as the value of v6.0A being in my windows registry under the following key: HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432NODE\Microsoft\Microsoft SDKs\Windows\v6.0A\ProductVersion.

Stripping leading and ending characters helped somehow. But the code seemed to contain various issues. The next one I met was in VersionSelector.cs, the GetInstalledVisualStudioVersions method:

foreach (Product product in products)
foreach (Product product in new List<Product>(products))
{
    float num;
    if (float.TryParse(product.SdkVersion, out num))
    {
        if (num < 8f)
        {
            products.Remove(product);
        }
    }
    else
    {
        products.Remove(product);
    }
}
return products;

Guess what a problem may raise up here? Who did write this?  You can't delete from a collection during looking it through by enumerator.

And at last for those having a non-english windows you have to add a line into the Program.cs file:


[STAThread]
private static void Main(string[] args)
{
    Thread.CurrentThread.CurrentCulture = CultureInfo.InvariantCulture;
 
    Mutex mutex = null;

It is required to parse numbers correctly.

You should place the compiled execution over the existing one to ensure that all its functionallity works properly. That because the program relies on files lying around.

If you somehow trust me, you can download the patched utility directly.

Monday, December 29, 2008

How to encrypt/decrypt with passphrase

I was stuck with this when I started using IIS7's AppCmd utility. The utility allows to export and import appication pool configuration, while not having an option to encode/decode identity password. But it was required to store the configuration under the source control system. So I decided to write a simple command-line encoding/decoding utility for a particular section being fetched by regular expression.

I use a symmetric algorithm because key is defined with passphrase for both encoding and decoding operations. Creating key from passphrase is based on hashing. We are required to have keys with fixed length and character sets, thus hashing goes a long way here:
var bytes = Encoding.Unicode.GetBytes(passphrase);
var key = SHA256Managed.Create().ComputeHash(bytes);
var iv = MD5.Create().ComputeHash(bytes);
Key and vector may have different requirements regarding length of byte arrays. For the RijndaelManaged class they are 32 and 16 respectively. Thus I use SHA256 and MD5 algorithms to get keys with the appropriate length.
var alg = SymmetricAlgorithm.Create();
var ms = new MemoryStream();
var buffer = Encoding.Unicode.GetBytes(text);
 
using (var enc = new CryptoStream(
    ms, alg.CreateEncryptor(key, iv),
    CryptoStreamMode.Write
))
{
    enc.Write(buffer, 0, buffer.Length);
}
The code above feeds the entire text to the CryptoStream. For passwords encoding it shouldn't be a perfomance issue. Decoding has quite similar implementation. Rather than using alg.CreateEncryptor it should use alg.CreateDecryptor there.

Additionally I use a couple of extention methods to format byte array to hexadecimal string and vice versa. Here are the entire Encode/Decode methods implementation and the helper methods:

private static string Encode(string text, string passphrase)
{
    var bytes = Encoding.Unicode.GetBytes(passphrase);
    var key = SHA256Managed.Create().ComputeHash(bytes);
    var iv = MD5.Create().ComputeHash(bytes);
 
    var alg = SymmetricAlgorithm.Create();
    var ms = new MemoryStream();
    var buffer = Encoding.Unicode.GetBytes(text);
 
    using(var enc = new CryptoStream(
        ms, alg.CreateEncryptor(key, iv),
        CryptoStreamMode.Write
    )) enc.Write(buffer, 0, buffer.Length);
 
    return ms.ToArray().ToHexString();
}
 
private static string Decode(string text, string passphrase)
{           
    var bytes = Encoding.Unicode.GetBytes(passphrase);
    var key = SHA256Managed.Create().ComputeHash(bytes);
    var iv = MD5.Create().ComputeHash(bytes);
 
    var alg = SymmetricAlgorithm.Create();
    var ms = new MemoryStream();
 
    var buffer = text.ToByteArray();
 
    try
    {
        using (var enc = new CryptoStream(
            ms, alg.CreateDecryptor(key, iv),
            CryptoStreamMode.Write
        )) enc.Write(buffer, 0, buffer.Length);
    }
    catch (Exception)
    {
        Console.Error.WriteLine("Error: wrong passphrase.");
        Environment.Exit(2);
    }
 
    return Encoding.Unicode.GetString(ms.ToArray());
}
...
internal static string ToHexString(this byte[] bytes)
{
    StringBuilder builder = new StringBuilder(3 * bytes.Length);
 
    for (int i = 0; i < bytes.Length; i++)
    {
        builder.AppendFormat("{0:x2}", bytes[i]);
    }
 
    return builder.ToString().ToLowerInvariant();
}
 
internal static byte[] ToByteArray(this string hexString)
{
    byte[] buffer = new byte[hexString.Length / 2];
 
    for (int i = 0; i < hexString.Length; i += 2)
    {
        buffer[i / 2] = byte.Parse(hexString.Substring(i, 2), NumberStyles.HexNumber);
    }
   
    return buffer;
}

Tuesday, December 9, 2008

How to add support for Google Chrome in ASP.NET

Do you like Google Chrome? I do, especially after they amended the section 11.4 of the EULA

Then what about adding browser capabilities for the browser in ASP.NET. It is better not to modify the browser definition files under the  <windir>\Microsoft.NET\Framework\<ver>\CONFIG\Browsers folder since they can be changed in a next .net framework release or service pack. Just add a .browser file to your web project under the App_Browser folder.

 Here is the content of the file:


<browsers>
  <browser id="GoogleChrome" parentID="Safari1Plus">
    <identification>
      <userAgent match="Chrome/(?'version'(?'major'\d+)\.(?'minor'\d+\.\d+).\d+)" />
    </identification>
    <capture>
    </capture>
    <capabilities>
      <capability name="browser"        value="Chrome" />
      <capability name="majorversion"   value="${major}" />
      <capability name="minorversion"   value="${minor}" />
      <capability name="version"        value="${version}" />
    </capabilities>
  </browser>
</browsers>

When I had done it, I found that it appeared a strange behavior. First starting the site in the Chrome made it stick with the Chrome definition for Apple Safari and vise versa.

After playing around with different variations of the above config I decided to stop wasting my time on it and to find how ASP.NET caches the browser capabilities using a reflector. The code of all this is pretty easy. The framework uses the User-Agent request-header as a cache key trimming its length by a configurable parameter.


 
<system.web>
    <browserCaps userAgentCacheKeyLength="128" />

I extended the value up to 128 from its default of 64 because the Chrome's user-agent string length is 118.

 

.net structures performance tips and tricks

Recently I dug into clr structure performance and found it's rather funny. Some revelations were new to me.

At first. If you have a structure containing two boolean fields, it is five times faster than structure containing three boolean fields, but it is equal to four boolean fields one! I merely return structure from method several million times and capture elapsed time by Stopwatcher (it is common scenario when we pass structure to method or return from it):
public struct TestStruct
{
    public bool Field1;
    public bool Field2;
    public bool Field3;
    public bool Field4;

    public static TestStruct CreateNew()
    {
        return default(TestStruct);
    }
}

static void Main(string[] args)
{
    const int iterations = 50000000;
    var sw = Stopwatch.StartNew();

    for (int i = 0; i < iterations; i++)
    {
        var a = TestStruct.CreateNew();
    }

    sw.Stop();
    Console.WriteLine("elapsed {0} ms", sw.ElapsedMilliseconds);
    Console.ReadKey();
}
It seems like it depends on how optimal structure layout is for clr. A boolean field takes one byte. Let's add one more boolean field (fifth). Now we have the five times slump again. But, if we add one more boolean field (sixth), we have the same loss in performance. Seventh such field makes performance seven times slower :).

Conclusionthe optimal structure size values are 1, 2, 4, 8. Let's take a look at another example:


public bool Field1;
public bool Field2;
public bool Field3;
public bool Field4;
public bool Field5;
public short Field6;
public bool Field7;


public short Field1;
public bool Field2;
public bool Field3;
public bool Field4;
public bool Field5;
public bool Field6;
public bool Field7;

The structure fields ordering from the first column four times slower than from the second. It seems it is because Field6 allocation starts from an uneven byte 5. Conclusion: field starting byte should be even if the field takes more than one byte.

The structure from our last example has eight bytes size and it's equal to a single long type allocation in memory. What if we create a 9 bytes size structure. Yes, it would be slowly and it is four times slowly. But the even and uneven allocation rules are still correct and can hit performace in approximately 2 times.

Can we make up more generalize conclusion? I think so. If we create a structure with a single long type field and then will add one more long field and then one more we will see that we have approximately arithmetical progression in growth of elapsed intereval for our loop. Conclusion: the less structure the better. Difference between one and two longs allocation is highest: five times, but then it becomes less and less for 2 vs 3, 3 vs 4 long type fields structures.

Comparing a simple class object creation time with 2 long type fields structure shows that such the structure is still 3 times faster. But a six long type fields structure creation time is approximately equal to that of the class.