Cryptography .NET, Avoiding Timing Attack

When comparing two MACs or hashes (also password hashes) for equality you would first think a simple comparison would be okay?  Think again as they are susceptible to timing attacks.

In cryptography, a timing attack is a side channel attack in which the attacker attempts to compromise a cryptosystem by analysing the time taken to execute cryptographic algorithms.

If you want to read more about this take a look at this article.

if ($hash1 == $hash2) {
//mac verification is Okay
return "hashs are equal"
} else {
//something happened
return "hashs verification failed!";
}

Okay, so what is wrong with this?

Both arguments must be of the same length to be compared successfully. When arguments of differing length are supplied, FALSE is returned immediately and the length of the known string may be leaked in the case of a timing attack.  It is done by timing the amount of time it takes to compare a known string length with the HASH, and as you will then find the length of the string based on the time it takes during the comparison.

Compares two-byte arrays in length-constant time. This comparison method is used so that password hashes cannot be extracted from on-line systems using a timing attack and then attacked off-line.

 private static bool SlowEquals(byte[] a, byte[] b)
 {
 uint diff = (uint)a.Length ^ (uint)b.Length;
 for (int i = 0; i < a.Length && i < b.Length; i++)
 diff |= (uint)(a[i] ^ b[i]);
 return diff == 0;
 }

What does the line diff |= (uint)(a[i] ^ b[i]); do?

This sets diff based on whether there’s a difference between a and b.

It avoids a timing attack by always walking through the entirety of the shorter of the two of a and b, regardless of whether there’s a mismatch sooner than that or not.

The diff |= (uint)(a[i] ^ (uint)b[i]) takes the exclusive-or of a byte of a with a corresponding byte of b. That will be 0 if the two bytes are the same, or non-zero if they’re different. It then ors that with diff.

Therefore, diff will be set to non-zero in an iteration if a difference was found between the inputs in that iteration. Once diff is given a non-zero value at any iteration of the loop, it will retain the non-zero value through further iterations.

Therefore, the final result in diff will be non-zero if any difference is found between corresponding bytes of a and b, and 0 only if all bytes (and the lengths) of a and b are equal.

Unlike a typical comparison, however, this will always execute the loop until all the bytes in the shorter of the two inputs have been compared to bytes in the other. A typical comparison would have an early-out where the loop would be broken as soon as a mismatch was found:

Microsoft provides a SecureString to hold sensitive data, and this uses a generic method Equals(Object), which determines whether the specified object is equal to the current object. There is no information saying that Equals method protects against Timing Attacks. Therefore we can only conclude that it is not safe for this operation.

I would suggest research into writing our own String Compare that is protected against Timing Attacks.

I’m not sure about all of this and further research is required, I did find this article which would be worth considering:

https://paragonie.com/blog/2015/11/preventing-timing-attacks-on-string-comparison-with-double-hmac-strategy

SecureString

I’ve come across a class in the System.Security namespace that is often overlooked. The class is SecureString. In this post, I will go over what SecureString is and why it is needed.

Have you ever come across these scenarios before?

  • A password appears in a log file accidentally.
  • A password is being shown at somewhere – once a GUI did show a command line of application that was being run, and the command line consisted of the password.
  • Using memory profiler to profile software with your colleague. Colleague sees your password in memory.
  • Using RedGate software that could capture the “value” of local variables in case of exceptions, amazingly useful. Though, I can imagine that it will log “string passwords” accidentally.
  • A crash dump that includes string password.

Do you know how to avoid all these problems? SecureString. It makes sure you don’t make silly mistakes as such. How does it avoid it? By making sure that password is encrypted in unmanaged memory and the real value can be only accessed when you are 90% sure what you’re doing.

In the sense, SecureString works pretty easily:

  1. Everything is encrypted
  2. User calls AppendChar
  3. Decrypt everything in UNMANAGED MEMORY and add the character
  4. Encrypt everything again in UNMANAGED MEMORY.

What if the user has access to your computer? Would a virus be able to get access to all the SecureStrings? Yes. All you need to do is hook yourself into RtlEncryptMemory then decrypt the memory, and you will get the location of the unencrypted memory address, and read it out. Voila! In fact, you could make a virus that will continuously scan for usage of SecureString and log all the activities with it. I am not saying it will be an easy task, but it can be done. As you can see, the “powerfulness” of SecureString is completely gone once there’s a user/virus on your system.

You have few points in your post. Sure, if you use some of the UI controls that hold “string password” internally, using actual SecureString is not that useful.

The bottom line is; if you have sensitive data(passwords, credit cards, ..), use SecureString. This is what C# Framework is following. For example, NetworkCredential class stores password as SecureString. If you look this, you can see over ~80 different usages in .NET framework of SecureString.

There are many cases when you have to convert SecureString to string because some API expects it.

The usual problem is either:

  • The API is GENERIC. It does not know that there’s a sensitive data.
  • The API knows that it’s dealing with sensitive data and uses “string” – that’s just bad design.

You raised a good point: what happens when you convert SecureString to a string? That can only happen because of the first point. Eg the API does not know that it’s sensitive data. I have personally not seen that happening. Getting string out of SecureString is not that simple.

It’s not simple for a simple reason; it was never intended to let the user convert SecureString to string, as you stated: GC will kick in. If you see yourself doing that, you need to step back and ask yourself: Why am I even doing this, or do I need this, why?

You can always extend the SecureString class with an extension method, such as ToEncryptedString(__SERVER__PUBLIC_KEY), which gives you a string instance of SecureString that is encrypted using server’s public key. The only server can then decrypt it. Problem solved, GC will never see the “original” string, as you never expose it in managed memory. This is precisely what is being done in PSRemotingCryptoHelper (EncryptSecureStringCore(SecureString secureString)).

And as something very almost-related: Mono SecureString does not encrypt at all. The implementation has been commented out because ..wait for it. “It somehow causes nunit test breakage”, which brings to my last point:

Not supported everywhere is SecureString. If the platform/architecture does not support SecureString, you’ll get an exception. The recommended list of deployment platforms is documentation.

Using SecureString for Sensitive Data

The standard System.String class has never been a very secure solution for storing sensitive strings such as passwords or credit card numbers. Using a string for this purposes has numerous problems including it’s not pinned, so the garbage collector can move it around and will leave several copies in memory, it’s not encrypted, so anyone who can read your processor’s memory will be able to see the values of the string easily. Also, if your process gets swapped out to disk, the unencrypted contents of the string will be sitting in your swap file. And it’s not mutable, so whenever you need to modify it there will be an old version and the new version both in memory.

Since it’s not mutable, there’s no effective way to clear it out when you’re done using it. Secure strings are held in encrypted memory by the CLR using the Data Protection API, or DPAPI, and they’re only unencrypted when they are accessed.  This limits the amount of time that your string is in plain text for an attacker to see.

The garbage collector will not move the encrypted string around in memory, so you never have to worry about multiple copies of your string sitting in your address space, unless you make copies of those.

SecureString also implements IDisposable, and when it’s disposed of or finalised if you forget to dispose of it, the memory that was used to hold your encrypted string will be zeroed out.

They also provide a feature that lets you lock them down as read only preventing other code from modifying your string.

You can create a secure string with a pointer to a character array and a length of that array. When constructed this way, the secure string will make a copy of your array, allowing you to zero out your insecure copy.

A secure string can also be constructed without an existing character array, and the data can be copied in one character at a time.

One important thing to note though is that SecureString shouldn’t be used as a blanket replacement for System.String, it should only be used in places where you need to store sensitive information that you do not want to spread around in memory for longer than is required.

To add data or modify data in your string, standard operations are provided. For instance, you’ll find an AppendChar, InsertAt, RemoveAt, and SetAt methods. MakeReadOnly and IsReadOnly allows you to lock down the secure string. Clear, Dispose, and the finaliser takes care of removing any trace of the safe string from memory.

The main idea with SecureString is that you would never store a password or other textual secret in plain text. Unfortunately, SecureString was introduced into the framework only after plenty of APIs were built and shipped using passwords stored in a string, such as the System.Net.NetworkCredential. So any application that must use these APIs had no option but to convert secure strings to strings.

However, the SecureString class itself doesn’t provide any method to get back a plain string with its content, precisely to discourage this type of usage. What a developer has to do is use functions from the System.Runtime.InteropServices.Marshal to get a native buffer with the plain string, marshal a value into managed string, and then very importantly free the native buffer. The best implementation to get a string out of a secure string is to use a try/finally block to free the native buffer.

Here is a sample application showing how you can use SecureString, please remember in production do not use the SecureString to String as this just does not make sense and would make it insecure.

SecureString

Here is a short method to convert a string to a SecureString

public static SecureString ToSecureString(this string source)
{
 if (string.IsNullOrWhiteSpace(source))
 return null;
 else
 {
 SecureString result = new SecureString();
 foreach (char c in source.ToCharArray())
 result.AppendChar(c);
 return result;
 }
}

Raising Multiple Exceptions with AggregateException

There are occasions where you are aware of many exceptions within your code that you want to raise together in one go. Perhaps your system makes a service call to a middleware orchestration component that potentially returns many exceptions detailed in its response, or another scenario might be a batch processing task dealing with many items in one process that require you to collate all exceptions until the end and then throw them together.

Let us look at the batch scenario in more detail. In this situation, if you raised the first exception that you found it would exit the method without processing the remaining items. Alternatively, you could store the exception information in a variable of some sort and once all the elements are processed using the information to construct an exception and throw it. While this approach works, there are some drawbacks. There is the extra effort required to create a viable storage container to hold the exception information, and this may mean modifying existing code not to throw an exception but instead to log the details in this new ‘exception detail helper class’. This solution also lacks the additional benefits you get with creating an exception then, for example, the numerous intrinsic properties that exist within Exception objects that provide valuable additional context information to support the message within the exception. Even when all the relevant information has been collated into a single exception class, then you are still left with one exception holding all that information when you may need to handle the exceptions individually and pass them off to existing error handling frameworks which rely on a type deriving from Exception.

Luckily included in .Net Framework 4.0 is the simple but very useful AggregateException class which lives in the System namespace (within mscorlib.dll). It was created to use the Task Parallel Library, and its use within that library is described on MSDN here. Don’t think that is it’s only used though, as it can be put to good use within your code in situations like those described above where you need to throw many exceptions, so let’s see what it offers.

The AggregateException class is an exception type, inheriting from System.Exception, that acts a wrapper for a collection of child exceptions. Within your code, you can create instances of any exception based type and add them to the AggregateException’s collection. The idea is a simple one, but the AggregateException’s beauty comes in the implementation of this simplicity. As it is a regular exception class, it can handle in the usual way by existing code but also as a unique exception collection by the particular system that cares about all the exceptions nested within its bowels.

The class accepts the child exceptions on one of its seven constructors and then exposes them through its InnerExceptions property. Unfortunately, this is a read-only collection, and so it is not possible to add inner exceptions to the AggregateException after it has instantiated (which would have been nice) and so you will need to store your exceptions in a collection until you’re ready to create the Aggregate and throw it:

// create a collection container to hold exceptions List<Exception> exceptions = new List<Exception>(); // do some stuff here ........ // we have an exception with an innerexception, so add it to the list exceptions.Add(new TimeoutException("It timed out", new ArgumentException("ID missing"))); // do more stuff ..... // Another exception, add to list exceptions.Add(new NotImplementedException("Somethings not implemented")); // all done, now create the AggregateException and throw it AggregateException aggEx = new AggregateException(exceptions); throw aggEx;

The method you use to store the exceptions is up to you as long as you have them all ready at the time you create the AggregateException class. Seven constructors are allowing you to pass combinations of nothing, a string message, collections or arrays of inner exceptions.

Once created you interact with the class as you would any other exception type:

try {    // do task } catch (AggregateException ex) {    // handle it  }

The key as it means that you can make use of existing code and patterns for handling exceptions within your (or third parties) codebase.

In addition to the general Exception members, the class exposes a few custom ones. The common InnerException property is there for compatibility, and this appears to return the first exception added to the AggregateException class via the constructor, so in the example above it would be the TimeoutException instance. All of the child exceptions expose via the InnerExceptions read-only collection property (as shown below).

The Flatten() method is another custom property that might prove useful if you find the need to nest Exceptions as inner exceptions within several AggregateExceptions. The method will iterate the InnerExceptions collection, and if it finds AggregateExceptions nested as InnerExceptions, it will promote their child exceptions to the parent level. As you can see in this example:

AggregateException aggExInner = 
 new AggregateException("inner AggEx", new TimeoutException());
AggregateException aggExOuter1 = 
 new AggregateException("outer 1 AggEx", aggExInner);
AggregateException aggExOuter2 = 
 new AggregateException("outer 2 AggEx", new ArgumentException());
AggregateException aggExMaster =
 new AggregateException(aggExOuter1, aggExOuter2);

If we create this structure above of AggregrateExceptions with inner exceptions of TimeoutException and ArgumentException then the InnerExceptions property of the parent AggregateException (i.e. aggExMaster) shows, as expected, two objects, both being of type AggregrateException and both containing child exceptions of their own:

But if we call Flatten()…

AggregateException aggExFlatterX = aggExMaster.Flatten();

…we get a new ArgumentException instance returned that contains still two objects, but this time the AggregrateException objects have gone, and we have the two child exceptions of TimeoutException and ArgumentException:

A useful feature to discard the AggregateException containers (which are effectively just packaging) and expose the real meat, i.e. the real exceptions that have been thrown and needs to be addressed.

If you’re wondering how the ToString() is implemented then the aggExMaster object in the examples above (without flattening) produces this:

System.AggregateException: One or more errors occurred. ---> System.AggregateException
: outer 1 AggEx ---> System.AggregateException: inner AggEx ---> 
System.TimeoutException: The operation has timed out. --- End of inner exception 
stack trace --- --- End of inner exception stack trace --- --- End of inner exception 
stack trace ------> (Inner Exception #0) System.AggregateException: outer 1 AggEx ---> 
System.AggregateException: inner AggEx ---> System.TimeoutException: The operation
 has timed out. --- End of inner exception stack trace --- --- End of inner 
exception stack trace ------> (Inner Exception #0) System.AggregateException: inner
AggEx ---> System.TimeoutException: The operation has timed out. --- End of inner 
exception stack trace ------> (Inner Exception #0) System.TimeoutException: The 
operation has timed out.<---<---<------> (Inner Exception #1) System.AggregateException
: outer 2 AggEx --- System.ArgumentException: Value does not fall within the expected
 range. --- End of inner exception stack trace ------> (Inner Exception #0) 
System.ArgumentException: Value does not fall within the expected range.

As you can see the data has been formatted in a neat and convenient way for readability, with separators between the inner exceptions.

In summary, this is a very useful class to be aware of and have in your arsenal whether you are dealing with the Parallel Tasks Library or you just need to manage multiple exceptions. I like simple and neat solutions, and to me, this is a good example of that philosophy.

Orginal Article

Using MongoDB

If you want to setup MongoDB with C#, then the best option is to follow the getting started guide on the MongoDB website

https://docs.mongodb.com/getting-started/csharp/client/

You can install the NuGet package for MongoDB by typing into the Package Manager Console

  • “Install-Package MongoDB.Driver”

Although MongoDB does provide an example that you can work through, I did find that the MoveNextAsync command is no longer supported and the case fails.

I built a simple application that uses MongoDB CRUD application that works very well to show how it can be implemented.

MongoDB

Here is a helpful MongoDB tool to administer your Mongo database, Robomogo

https://robomongo.org

On my searching to find good source of material I came across this nice video demo here https://youtu.be/ubwC63DwJ8w

Custom Configuration in your config

This post is to show how to setup a customised configuration section in your app.config or web.conifg of your .NET application

I have written an article on this before, back in 2008, but things have moved on in the libraries and this is the latest and easiest way to create your own customised configuration.

We just need to define our class, inherit from System.Configuration.ConfigurationSection, and add a property per setting we wish to store.

using System;
using System.Configuration;

public class BlogSettings : ConfigurationSection
{
 private static BlogSettings settings = ConfigurationManager.GetSection("BlogSettings") as BlogSettings;
 
 public static BlogSettings Settings
 {
 get
 {
 return settings;
 }
 }

 [ConfigurationProperty("frontPagePostCount" , DefaultValue = 20 , IsRequired = false)]
 [IntegerValidator(MinValue = 1 , MaxValue = 100)]
 public int FrontPagePostCount
 {
 get { return (int)this["frontPagePostCount"]; }
 set { this["frontPagePostCount"] = value; }
 }


 [ConfigurationProperty("title" , IsRequired=true)]
 public string Title
 {
 get { return (string)this["title"]; }
 set { this["title"] = value; }
 }
}

Notice that you use an indexed property to store and retrieve each property value.

Also not a static property named Settings for convenience.

Add your new configuration section to web.config (or app.config).

<configuration>
 <configSections>
 <section name="BlogSettings" type="Fully.Qualified.TypeName.BlogSettings, 
 AssemblyName" />
 </configSections>
 <BlogSettings
 frontPagePostCount="10"
 title="You’ve Been Haacked" />
</configuration>

And to access the new configuration using code it is as simple as:

string title = BlogSettings.Settings.Title;
Response.Write(title); //it works!!!

Here is the source code: Configuration Example

Another article that goes into this in a lot more details can be found on Code Project – Unraveling the Mysteries of .NET 2.0 Configuration

How slow are Exceptions

I’ve seen from time to time applications that wrap a TRY – CATCH around some code, but do nothing except carry on.  So why is this so wrong?

As programmers we want to write quality code that solves problems. Unfortunately, exceptions come as side effects of our code. No one likes side effects, so we soon find our own ways to get around them. I have seen some smart programmers deal with exceptions the following way:

public void consumeAndForgetAllExceptions(){
    try {
        ...some code that throws exceptions
    } catch (Exception ex){
        ex.printStacktrace();
    }
}

What is wrong with the code above?

Once an exception is thrown, normal program execution is suspended and control is transferred to the catch block. The catch block catches the exception and just suppresses it. Execution of the program continues after the catch block, as if nothing had happened.

How about the following?

public void someMethod() 
 {
 throw new Exception();
 }

This method is a blank one; it does not have any code in it. How can a blank method throw exceptions? C# does not stop you from doing this. Recently, I came across similar code where the method was declared to throw exceptions, but there was no code that actually generated that exception. When I asked the programmer, he replied “I know, it is corrupting the API, but I am used to doing it and it works.”

This debate goes around in circles in the C# community. I have seen several C# programmers struggle with the use of exceptions. If not used correctly, exceptions can slow down your program, as it takes memory and CPU power to create, throw, and catch exceptions. If overused, they make the code difficult to read and frustrating for the programmers using the API. We all know frustrations lead to hacks and code smells. The client code may circumvent the issue by just ignoring exceptions or throwing them.

Let’s go back to basics and start with the definition of an Exception:

Exception handling is the process of responding to the occurrence, during computation, of exceptions – anomalous or exceptional conditions requiring special processing – often changing the normal flow of program execution. It is provided by specialized programming language constructs or computer hardware mechanisms.

The important words here I feel is special processing something which is out of your control.

Here is a simple example

 var a = 0;
 int b;
 try
 {
  b = 10 / a;
 }
 catch
 {
 }

So what is wrong with this?

I call it lazy and poormans programming, one correct solution to this could be:

 var a = 0;
 int b;
 if (a != 0)
  b = 10 / a;

Okay this is a very simple example of abusive Exception handling, but does it really matter?

I wrote a small benchmark test to see just what the difference is in performance.  Just a simple 10,000 parallel loop count running the same code over and over again.

I was quite overwhelmed with the performnce hit of the Exception handling, even just in this small example.  The Exception handling caused the code to slow down by over 600 times.

600 times slower code, well that does not matter if this is only happening once I hear you say…!

The performance is just one area, what about the extra memory that the exception is using.  With a few changes to the Benchmark application am able to monitor the Garbage Collector, which although it is not a true representation of how much memory is being used it will provide a very good gauge.

The results of the memory useage were quite staggering

  • 14,416 for the programming logic
  • 2,501,832 for the exception handing

Just another reason why this is not acceptable to use the exception handler in this way.

This is just a short article on Excpetion Handling, but it does frustrate me when I see Exception Handling  being abused.

ExceptionPerformance

Object to CSV extension

Here is a nice little extension that will take an object and convert it into a CSV file.

Also has the option if you want to include a header on the CSV file.

public static class ObjectToCsv
 {
 public static string ToCsv<T>(this IEnumerable<T> items, bool includeHeading = true)
 where T : class
 {
 var csvBuilder = new StringBuilder();
 var properties = typeof(T).GetProperties();

 if (includeHeading)
 {
 csvBuilder.AppendLine(string.Join(",", properties.Select(p => p.Name.ToCsvValue()).ToArray()));
 }

 foreach (var item in items)
 {
 var line = string.Join(",", properties.Select(p => p.GetValue(item, null).ToCsvValue()).ToArray());
 csvBuilder.AppendLine(line);
 }
 return csvBuilder.ToString();
 }

 private static string ToCsvValue<T>(this T item)
 {
 if (item == null) return "\"\"";

 if (item is string)
 {
 return $"\"{item.ToString().Replace("\"", "\\\"")}\"";
 }
 double dummy;
 return double.TryParse(item.ToString(), out dummy) ? $"{item}" : $"\"{item}\"";
 }
 }

Upgrading an ASP.NET MVC 3 Project to ASP.NET MVC 4

 

ASP.NET MVC 4 can be installed side by side with ASP.NET MVC 3 on the same computer, which gives you flexibility in choosing when to upgrade an ASP.NET MVC 3 application to ASP.NET MVC 4.

The simplest way to upgrade is to create a new ASP.NET MVC 4 project and copy all the views, controllers, code, and content files from the existing MVC 3 project to the new project and then to update the assembly references in the new project to match any non-MVC template included assembiles you are using. If you have made changes to the Web.config file in the MVC 3 project, you must also merge those changes into the Web.config file in the MVC 4 project.

To manually upgrade an existing ASP.NET MVC 3 application to version 4, do the following:

In all Web.config files in the project (there is one in the root of the project, one in the Views folder, and one in the Views folder for each area in your project), replace every instance of the following text (note: System.Web.WebPages, Version=1.0.0.0 is not found in projects created with Visual Studio 2012):

System.Web.Mvc, Version=3.0.0.0
System.Web.WebPages, Version=1.0.0.0
System.Web.Helpers, Version=1.0.0.0
System.Web.WebPages.Razor, Version=1.0.0.0

with the following corresponding text:

System.Web.Mvc, Version=4.0.0.0
System.Web.WebPages, Version=2.0.0.0
System.Web.Helpers, Version=2.0.0.0
System.Web.WebPages.Razor, Version=2.0.0.0

In the root Web.config file, update the webPages:Version element to “2.0.0.0” and add a new PreserveLoginUrl key that has the value “true”:

<appSettings>
  <add key="webpages:Version" value="2.0.0.0" />
  <add key="PreserveLoginUrl" value="true" />
</appSettings>

n Solution Explorer, right-click on the References and select Manage NuGet Packages. In the left pane, select Online\NuGet official package source, then update the following:

  • ASP.NET MVC 4
  • (Optional) jQuery, jQuery Validation and jQuery UI
  • (Optional) Entity Framework
  • (Optonal) Modernizr

In Solution Explorer, right-click the project name and then select Unload Project. Then right-click the name again and select Edit ProjectName.csproj.

Locate the ProjectTypeGuids element and replace {E53F8FEA-EAE0-44A6-8774-FFD645390401} with {E3E379DF-F4C6-4180-9B81-6769533ABE47}.

Save the changes, close the project (.csproj) file you were editing, right-click the project, and then select Reload Project.

If the project references any third-party libraries that are compiled using previous versions of ASP.NET MVC, open the root Web.config file and add the following three bindingRedirect elements under the configuration section:

<configuration>
  <!--... elements deleted for clarity ...-->
 
  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="System.Web.Helpers" 
             publicKeyToken="31bf3856ad364e35" />
        <bindingRedirect oldVersion="1.0.0.0" newVersion="2.0.0.0"/>
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="System.Web.Mvc" 
             publicKeyToken="31bf3856ad364e35" />
        <bindingRedirect oldVersion="1.0.0.0-3.0.0.0" newVersion="4.0.0.0"/>
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="System.Web.WebPages" 
             publicKeyToken="31bf3856ad364e35" />
        <bindingRedirect oldVersion="1.0.0.0" newVersion="2.0.0.0"/>
      </dependentAssembly>
    </assemblyBinding>
  </runtime>
</configuration>

Original article can be found at: MVC 4 release notes