Sunday, April 30, 2017 #

SQL server AlwaysOn Availability Group data latency issue on secondary replicas and synchronous state monitoring

This article explains how data synchronization works on SQL Server AlwaysOn Availability Group and some details about how to use sys.dm_hadr_database_replica_states table to check replica states.


I borrowed this daligram from this article which explains data synchronization process on SQL Server AlwaysOn Availability group.



The article has full details about the process. Things are worthing noting here are step 5 and 6.
When a transaction log is received by a secondary replica, it will be cached and then

  • 5. Log is hardened to the disk. Once the log is hardened, an acknowledgement is sent back to the primary replica.

  • 6.Redo process will write the change to the actual database


So for a Synchronous-Commit secondary replica, after step 5, the primary replica will be acknowledged to complete the transaction as “no data loss” has been confirmed on the secondary replica. This means after a transaction is completed, SQL server only guarantees the update has been written to the secondary replica’s transaction logs files rather than the actual data file. So there will be some data latency on the secondary replicas even though they are configured as “ Synchronous-Commit”. This means after you made some changes on the primary replica, if you try to read it immediately from a secondary replica, you might find your changes are there yet.




This is why in our project, we need to monitor the data synchronization states on the secondary replicas. To get the replica states, I use this query:


select  

   r.replica_server_name,

   rs.is_primary_replica IsPrimary,

   rs.last_received_lsn,

   rs.last_hardened_lsn,

   rs.last_redone_lsn,

   rs.end_of_log_lsn,

   rs.last_commit_lsn

from sys.availability_replicas r

inner join sys.dm_hadr_database_replica_states rs on r.replica_id = rs.replica_id


The fields end with “_lsn” are the last Log Sequence Number at different stages.


  • last_received_lsn: the last LSN a secondary replica has received

  • end_of_log_lsn: the last LSN a has been cached

  • last_hardened_lsn: the last LSN a has been hardened to disk

  • last_redone_lsn: the last LSN a has been redone

  • last_commit_lsn: not sure what exactly this one this. But from my test, most of the time, it equals to last_redone_lsn with very rare cases, it is little smaller than last_redone_lsn. So I guess it happens a little bit after redone.




If you run the query on the primary replica, it will returns the information for all replicas


If you run the query on a secondary replica, it will returns the data for itself.



Now we need to understand the format of those LSNs. As this article explained, a LSN is in following format:


  • the VLF (Virtual Log Files) sequence number 0x14 (20 decimal)

  • in the log block starting at offset 0x61 in the VLF (measured in units of 512 bytes)

  • the slot number 1



As the LSN values returned by the query are in decimal, we will have to break them into parts like this:


Now we can compare the LSN values to figure out some information about state of the replica. For example, we can tell how much redo process in behind the logs have been hardened on Replica-2:


last_hardened_lsn - last_redone_lsn = 5137616 - 5137608 = 8.


Note, we don’t need to include the slot number(the last 5 digits) into the calculation. In fact, most of the LSNs in dm_hadr_database_replica_states table do not have slot number. As this document says: they are not actual log sequence numbers (LSNs), rather each of these values reflects a log-block ID padded with zeroes. We can tell this by looking at the last 5 digits of a LSN number. If it always ends with 00001, it means it is not actual LSN.


As from this article we know the block offset in measured in units of 512 bytes, in the example above, the difference between last_hardened_lsn and last_redone_lsn that means there is 8 * 512 = 4096 bytes data is waiting to be written to the data file.


UPDATE: I have just observed that last_redone_lsn could be NULL when there are no replicas acting as primary in the group. It is not always and I don't know under what condition it will NULL. However last_commit_lsn seems always have a value and as I mentioned it has similar value as last_redone_lsn(maybe just slightly behind last_redone_lsn). So  last_commit_lsn might be used when last_redone_lsn is not available. 


This article has much better explanation about how the transaction logs work in Availability group.

Posted On Sunday, April 30, 2017 9:58 AM | Comments (4)

Friday, March 3, 2017 #

Capture HTTPS traffic from iOS devices using Charles

This article will show you how to set up Charles and iOS devices so that you can capture HTTPS traffic from iOS devices using Charles. Step 0: Download and install Charles from https://www.charlesproxy.com/


Step 1: Find the IP address of your mac:


Ifconfig | grep “inet “     




Step 2: Find the port number Charles uses listens on. It should be 8888 if you haven’t change the default value. You can find the number from Settings-->Proxy Settings





Step 3: You need set up the proxy on the iOS device.

On the iOS device, Settings → Wi-Fi, click on the little i icon next to the Wi-Fi.




Change HTTP PROXY to Manual, type in the IP Address and Port number of your mac.



Step 4: Go back to Charles, set filters, so that Charles only record the connections your are interested. You don't have to do this step, in case you don't want Charles are flooded with records you are not interested as it will record everything.



Just include the urls you want to record.



Now, if you trigger HTTPS connections from your iOS app, you will find they are recorded by Charles, but the messages are encrypted. You can’t find any useful information from i.


Step 5: Enable SSL Proxying. Right click the URL, and check “Enable SSL Proxying”





You still won’t be able to get any useful information at this stage, as you iOS does not trust the certificate Charles provides.



Step 6: Install Charles’ certificate


Open Safari from your iOS device and browse to:  http://www.charlesproxy.com/getssl

Your iOS device will ask you to install the certificate. Just do it.



That is it. Now you should be able to all the details of the connections.







Posted On Friday, March 3, 2017 12:02 PM | Comments (8)

Execute AsyncTask like jquery’s $.ajax in Kotlin on Android

Last time I showed how to execute an AsyncTask in Jquery style. Now I will show how to do it in Kotlin.


Kotlin has much better support for functional programming, functions/lambdas are first class member of the language. A parameter of a method can be just typed function rather than having to be interface. So I don’t have to create interface for Func<T> and Action<T>.


This is how MyAsyncTask<T> looks like in Kotlin, note the type of task and completionHandler are ()->T or (T)->Unit?.

class MyAsyncTask<T>(private val task: () -> T) : AsyncTask<Void, Void, T>() {


  private var completionHandler: ((T) -> Unit)? = null

  private var errorHandler: ((Exception) -> Unit)? = null

  private var exception: Exception? = null


  fun setCompletionHandler(action: (T) -> Unit) {

      this.completionHandler = action

  }


  fun setErrorHandler(action: (Exception) -> Unit) {

      this.errorHandler = action

  }


  override fun doInBackground(vararg voids: Void): T? {

      try {

          return task.invoke()

      } catch (ex: Exception) {

          exception = ex

      }


      return null

  }


  override fun onPostExecute(result: T) {


      if (exception != null && errorHandler != null) {

          errorHandler!!.invoke(exception!!)

      } else if (completionHandler != null) {

          completionHandler!!.invoke(result)

      }

  }

}



And here is Async<T>:


class Async<T> private constructor() {


  private var task: MyAsyncTask<T>? = null


  fun whenComplete(action: (T) -> Unit): Async<T> {

      task!!.setCompletionHandler(action)

      return this

  }


  fun onError(action: (Exception) -> Unit): Async<T> {

      task!!.setErrorHandler(action)

      return this

  }


  fun execute() {

      try {

          task!!.execute()

      } catch (ex: Exception) {

          ex.printStackTrace()

      }

  }


  companion object {

      fun <T> run(task: () -> T): Async<T> {

          val runner = Async<T>()

          runner.task = MyAsyncTask(task)

          return runner

      }

  }

}


Here is an example of how it is used.


override fun onCreate(savedInstanceState: Bundle?) {

  super.onCreate(savedInstanceState)

  setContentView(R.layout.activity_main)


  Async.run {

      val api = MyAPI()

      api.signInUser("user_name", "password")

  }.whenComplete {

      //it.displayName

  }.onError {

      //it.message

  }.execute()

}


Posted On Friday, March 3, 2017 10:16 AM | Comments (4)

Wednesday, March 1, 2017 #

Execute AsyncTask like jquery’s $.ajax in Java on Android

On Android, if you want to make an async call, AsyncTask is the way to go. Basically you will have to implement something like this:


private class LongOperation extends AsyncTask<String, Void, String> {

  @Override

  protected String doInBackground(String... params) {

      //execute a long running operation

  }

  @Override

  protected void onPostExecute(String result) {

      //back to UI thread, can update UI

  }

}


Then you can call new LongOperation().execute("");


I don’t like having to write so much boilerplate code to just invoke a method asynchronously and I hate those mystical type parameters with AsyncTask. I want something simple, something like jquery’s $.ajax. I want write async code like this:


Async.run(() -> {

  //execute a long running operation

})

.whenComplete(result -> {

  //back to UI thread, can update UI

})

.onError(ex -> {

  //handle error, on UI thread

})


Here is how I do it.

First I need to create 2 interfaces:

public interface Action<T> {

  void execute(T param);

}


public interface Func<T> {

  T execute() throws Exception;

}


Then I need a wrapper on AsyncTask:


public class MyAsyncTask<T> extends AsyncTask<Void, Void, T> {


  private Func<T> task;

  private Action<T> completionHandler;

  private Action<Exception> errorHandler;

  private Exception exception;


  public MyAsyncTask(Func<T> task) {

      this.task = task;

  }


  public void setCompletionHandler(Action<T> action) {

      this.completionHandler = action;

  }


  public void setErrorHandler(Action<Exception> action) {

      this.errorHandler = action;

  }


  @Override

  protected T doInBackground(Void... voids) {

      try {

          return task.execute();

      } catch (Exception ex) {

          exception = ex;

      }

      return null;

  }


  @Override

  protected void onPostExecute(T result) {

      if (exception != null && errorHandler != null) {

          errorHandler.execute(exception);

      } else if (completionHandler != null) {

          completionHandler.execute(result);

      }

  }

}


Last, I need a small class to enable the fluent interface:


public class Async<T> {


  private  MyAsyncTask<T> task;


  private Async() {

  }


  public static <T> Async<T> run(Func<T> task) {

      Async<T> runner = new Async<T>();

      runner.task = new MyAsyncTask<T>(task);

      return runner;

  }


  public Async<T> whenComplete(Action<T> action) {

      task.setCompletionHandler(action);

      return this;

  }


  public Async<T> onError(Action<Exception> action) {

      task.setErrorHandler(action);

      return this;

  }


  public void execute() {

      try {

          task.execute();

      } catch (Exception ex) {

          ex.printStackTrace();

      }

  }

}



That is all I need!

Here is an example usage of it:


private void loadUser() {

  final Dialog progress = showProgressDialog();


  Async.run(new Func<UserInfo>() {

      @Override

      public UserInfo execute() {

          APIServer api = new APIServer()

          return api.loadUser();

      }

      })

      .whenComplete(new Action<UserInfo>() {

          @Override

          public void execute(UserInfo user) {

              progress.dismiss();

              //update user UI

              }

          }

      })

      .onError(new Action<Exception>() {

          @Override

          public void execute(Exception ex) {

              progress.dismiss();

              showAlertDialog(ex.getMessage());

          }

      })

      .execute();

}


Even simpler with lambda expression in Java 8!!!


In Java 8, an interface with single abstract method is a functional interface. A lambda expression with same signature (parameters and return type) as the interface method can be cast to the interface type automatically. As both Func and Action have only one method, they are functional interfaces. So we a method require a Func or Action as a parameter, we can pass in a lambda expression. If you want to know more about functional interface, this article is good to read.

So using lambda expression, we can make async call like this:


private void loadUser() {

  final Dialog progress = showProgressDialog();


  Async.run(() -> {

          APIServer api = new APIServer()

          return api.loadUser();

      })

      .whenComplete((user) -> {

          progress.dismiss();

          //update user UI

      })

      .onError((ex) -> {

progress.dismiss();

showAlertDialog(ex.getMessage());

      })

      .execute();

}


Posted On Wednesday, March 1, 2017 10:06 PM | Comments (24)

Monday, October 17, 2016 #

HttpModule event execution order for multiple modules

I did some tests to find out the event execution order for multiple HttpMoudles in ASP.NET. Here is the what I found:
When there are multiple HttpMoudles, and the modules are registered in Web.config,  all event handlers are executed in the SAME order in which they were registered, except PreSendRequestHeaders and PreSendRequestContent which are in REVERSE order.

Here is an example, suppose we have 3 modules registered in Web.config in following order.

    <modules>
      <add name="Module1" />
      <add name="Module2" />
      <add name="Module3" />
    </modules>

An event, for example BeginRequest, first Module1's BeginRequest will be executed then Module2's and the Module3's. 

This is true for all events except PreSendRequestHeaders  and PreSendRequestContent which are in following order:
So they are in REVERSE order and PreSendRequestContent  and PreSendRequestHeaders in the same module are executed one after another then next module.

In terms of events in one module, this is the execution order:
  • BeginRequest
  • AuthenticateRequest
  • PostAuthenticateRequest
  • AuthorizeRequest
  • PostAuthorizeRequest
  • ResolveRequestCache
  • PostResolveRequestCache
  • MapRequestHandler
  • PostMapRequestHandler
  • AcquireRequestState
  • PostAcquireRequestState
  • PreRequestHandlerExecute
  • PostRequestHandlerExecute
  • ReleaseRequestState
  • PostReleaseRequestState
  • UpdateRequestCache
  • PostUpdateRequestCache
  • LogRequest
  • PostLogRequest
  • EndRequest
  • PreSendRequestContent
  • PreSendRequestHeaders
  • RequestCompleted
This result pretty much confirms what this MSDN document states.

Here is full result from my tests:

BeginRequest
o   Moudle1
o   Moudle2
o   Moudle3
AuthenticateRequest
o   Moudle1
o   Moudle2
o   Moudle3
PostAuthenticateRequest
o   Moudle1
o   Moudle2
o   Moudle3
AuthorizeRequest
o   Moudle1
o   Moudle2
o   Moudle3
PostAuthorizeRequest
o   Moudle1
o   Moudle2
o   Moudle3
ResolveRequestCache
o   Moudle1
o   Moudle2
o   Moudle3
PostResolveRequestCache
o   Moudle1
o   Moudle2
o   Moudle3
MapRequestHandler
o   Moudle1
o   Moudle2
o   Moudle3
PostMapRequestHandler
o   Moudle1
o   Moudle2
o   Moudle3
AcquireRequestState
o   Moudle1
o   Moudle2
o   Moudle3
PostAcquireRequestState
o   Moudle1
o   Moudle2
o   Moudle3
PreRequestHandlerExecute
o   Moudle1
o   Moudle2
o   Moudle3
PostRequestHandlerExecute
o   Moudle1
o   Moudle2
o   Moudle3
ReleaseRequestState
o   Moudle1
o   Moudle2
o   Moudle3
PostReleaseRequestState
o   Moudle1
o   Moudle2
o   Moudle3
UpdateRequestCache
o   Moudle1
o   Moudle2
o   Moudle3
PostUpdateRequestCache
o   Moudle1
o   Moudle2
o   Moudle3
LogRequest
o   Moudle1
o   Moudle2
o   Moudle3
PostLogRequest
o   Moudle1
o   Moudle2
o   Moudle3
EndRequest
o   Moudle1
o   Moudle2
o   Moudle3
PreSendRequestContent
o   Moudle3
PreSendRequestHeaders
o   Moudle3
PreSendRequestContent
o   Moudle2
PreSendRequestHeaders
o   Moudle2
PreSendRequestContent
o   Moudle1
PreSendRequestHeaders
o   Moudle1
RequestCompleted
o   Moudle1
o   Moudle2
o   Moudle3

Final thought: It is not a good idea to have multiple http modules which rely on modules's execution order. Even though the order does seem to be predictable, it is error prone and hard to to debug. It may also have some performance cost. If there is modules which is dependent on another, it'd better just combine them into one.

Posted On Monday, October 17, 2016 11:06 AM | Comments (2)

Thursday, August 4, 2016 #

.NET thread-pool threads and CLR worker threads

I will try to answer following questions in this article:

  • How many threads are available in .NET thread pool?
  • How long does the CLR take to create a new Non-thread-pool thread?
  • How long does the CLR take to schedule a task on a thread-pool thread?
  • How long does the CLR take to create a new thread-pool thread when it does not have enough thread available in the pool?

 

How many threads are available in .NET thread pool?

Code:

Output:

 

The min number of workerThreads is 8. It is the number of threads available in the tread pool when you start a program. I will prove it with my testing later in this article.

 

How long does the CLR take to create a new Non-thread pool thread?

Code:

Output:

 

In the test, I simple create 100 CRL worker thread, in each tread, it just sleeps for 5 seconds and increase a counter. When counter values reaches to 100, means all threads completed, then the test stops.

The result shows it took 220 milliseconds to create all the threads, so in average, .NET takes about 2.2 milliseconds to create a worker thread.

The whole test took 5252 milliseconds to complete, as each thread sleeps for 5 seconds, and all thread run in parallel. So the numbers all add up. 

Note: 100 threads is a very small sample size, and how long exactly .NET will take to create a thread will vary on different hardware/test environment. The purpose of this article is just to give you some basic ideas of how expensive some operations are, rather than the accuracy. This will apply for the rest of the article.

 

How long does the CLR take to schedule a task on a thread-pool thread?

Code:

Output

 

Again, similarly test, just use thread-pool threads rather than creating our own threads this time. The result shows now it only takes 0.147 milliseconds for each thread to be scheduled, much faster compare to 2.2 milliseconds in last test. Just as expected, right? So far so good, or maybe, so far so boring…

Now, here comes the interesting part, this is the result for total amount of time to complete the test:

 

What?! It is much longer than the last test! Why?!

To show what’s going on, I have printed the thread ID in each worker thread. Here is what it looks like:

 

The first 8 threads were very quickly printed on the screen, pretty much at same time. Then starts from the 9th, the next 8 threads were pretty slow, each one took about 1 second to show on the screen. Then the most of the rest became very fast again, with just a few slower exceptions.

Here is the explanation: As we have showed in the beginning of this article, the min number of workerThreads is 8, and it must be the number of threads available in the thread pool when a program starts. That explains why the first 8 threads started running very fast. Then as the program requires more threads, and thread pool become empty, it starts to create new threads, so the next 8 threads started very slowly. It took about 0.5-1 seconds to create a thread, after the next 8 threads started which took about 5 seconds, and then the previously scheduled threads began to return back to the pool, starting new jobs became fast again until the thread pool is running out.

To prove this, I changed the min worker thread settings and ran the same test again:

The total time spent is just over 5 seconds now, no more time spent on creating new thread-pool thread.

 

Now we want to find out:

How long does the CLR take to create a new thread-pool thread when it does not have enough thread available in the pool?

Code:

Output

 

 Now, we want to find out the time .NET will take to create new thread-pool thread. So we can’t let the worker thread sleep anymore. I used ManualResetEvent to signal the worker threads to return. Hopefully, ManualResetEvent will take no time to trigger the return. But again, accuracy is not a big concern here.

The result shows it took 951.57 milliseconds to create a thread-pool thread.
Now, here comes a question, why it takes so long to create a thread-pool thread? My previous test has just showed it only takes 2.2 milliseconds to create a CLR worker thread? Why thread pool takes so long?

It turns out that the time it spends not only for creating a thread, rather there is some logic for optimising thread pool. It will make decision to create a new thread or wait for a thread to be available. This is from MSDN:

When a minimum is reached, the thread pool can create additional threads or wait until some tasks complete. Beginning with the .NET Framework 4, the thread pool creates and destroys worker threads in order to optimize throughput, which is defined as the number of tasks that complete per unit of time. Too few threads might not make optimal use of available resources, whereas too many threads could increase resource contention.

As most of the MSDN documents, it is not that clear on what it is trying to say. But I think it can be translated into simple English as

“When .NET thread pool is empty, it will only create a new thread for about every 0.5-1 seconds. “

 

Change machine.config for larger number of minWorkerthread

So for a program, if we know we are going to use a lot of threads from thread-pool, e.g. WCF service hosted by IIS, we better have larger number minWorkerThread. We can do this by changing machine.config. (%windir%\Microsoft.NET\Framework64\[version]\config\machine.config)

Here is an example

 <configuration>
 <system.net>
         <connectionManagement>
             <add address="*" maxconnection="24" />
         </connectionManagement>
     </system.net>
     <system.web>
         <processModel autoConfig="true"
             maxWorkerThreads = "100"
             maxIoThreads = "100"
             minWorkerThreads = "50"
             minIoThreads = "50"
         />
         <httpRuntime 
             minFreeThreads="176" 
             minLocalRequestFreeThreads="152" 
         />
     </system.web>
</configuration>

Note: the thread numbers in the config are per CPU. minWorkerThreads = "50", will let me to have 400 threads in the thread pool as my machine has 8 cores. That also explains the magic default minWorkerThreads value “8” (not Chinese lucky number!).  

 

Here is an article about more details of Tuning IIS - machine.config settings

 

Posted On Thursday, August 4, 2016 3:33 PM | Comments (27)

Saturday, November 29, 2008 #

Solving memory leak in javascript with try-finally

In my previous post, I discussed how return statement is executed in try-finally clause. So in following program:

            string str = "original string";

      try {

            return str;

      } finally {

            str = "changed in finally";

      }

 

the original value will be returned instead of the value changed in finally block. Would this feature be useful anywhere? Well, I can’t find anything in C#, but I can think up an example in javascript.

 

First, let’s have a look of following javascrip code:

            function createDiv() {

            var div = document.createElement("div");

            div.onclick = function () {  }

}

 

This looks quite ordinary javasript code, doesn’t it? But can you spot a potential memory leak in the code?

 

To prove there is a memory leak, let’s just open following html page in IE7:

<html>

  <body>

    <script type="text/javascript">

      function createDiv() {

        var div = document.createElement("div");

        div.onclick = function () {  }

      }

      for (var i = 0; i < 100000; i++) {

        createDiv();

      }

    </script>

  </body>

</html>

 

 

This is what happened after I refreshed the page a few times. For each refresh, the memory usage was increased by approximately 100M, and it would never go down. Even I went to another page or closed the tab page, the memory was still held by IE7. Shut down IE is the only way to reclaim the memory back! (Note: this memory leak won’t happen in firefox 3)

The cause of the memory leak is circular reference. As you may know, calling a javascript function will create an object.  This object is not accessible from your code, but it is used for javascript to maintain its scope chain. So, in the code above, when createDiv is called, a scope object is created which has a property “div” which points to a div object created by document.createEmelemt(“div”).  Then “div” object’s onclick property is assigned with a function object defined inside function “createDiv”.  As the function for onclick lives inside the closure, it has a reference of its parent scope object. So, have a look of the following picture, there is clearly a circular reference. That's why it causes memory leak.

 


So, how can we fix the problem? Of course, the easiest way is to assign div = null or move onclick function outside the createDiv like this:

 

function createDiv() {

            var div = document.createElement("div");

            div.onclick = function () {  }

            div = null;

}

 

or

 

function createDiv() {

            var div = document.createElement("div");

            div.onclick = handleClick;

      }

function handleClick() {};

 

But what if onclick need to access a variable defined in createDiv and you want to return div from createDiv? Now, try-finally will help.

 

function createDiv() {

            var div = document.createElement("div");

            var greeting = "hello world!";

            div.onclick = function() { alert(greeting); };

            try {

                  return div;

            } finally {

                  div = null;

            }

      }

 

Now, in the try block, div is returned properly, and the link between createDiv scope object and div object is guaranteed to be broken by assigning createDiv’div variable (note, it is just the div variable, not the div object itself) to null in the finally block.

Posted On Saturday, November 29, 2008 11:59 AM | Comments (5)

Friday, November 28, 2008 #

Code smell – many usings

If you see some code like this, you know there is something very wrong.

 

 

Why on earth should I have to deal with System.Xml, System.Threading, System.IO, System.Diagnostics, System.Runtime.InteropServices, System.Security.Cryptography, System.Net, System.Net.Sockets in a Form class?

Posted On Friday, November 28, 2008 9:55 AM | Comments (3)

Thursday, November 27, 2008 #

C# quiz - try-finally

So what is the output for following program?

public class SimpleTest

{

      public static void Main(string[] args) {

            Console.WriteLine(GetString());

            Console.ReadLine();

      }

     

      private static string GetString() {

            string str = "original string";

            try {

                  return str;

            } finally {

                  str = "changed in finally";

            }

      }

}

It looks very simple, but most of my developer friends and I all had the wrong answer. In the college, we were all told that finally block is guaranteed to be executed before a method returns, right? So it should display “changed in finally”, shouldn’t it? Unfortunately, this is a wrong answer. The program will actually print “original string”!

So, why? The answer is pretty simple. Though “return str;” is just one statement in C#, it is made of a few IL instructions. So if we have a look the complied IL code, the program first loads the “original string” and stores it in location0, then in the try block, it loads the value from lcoation0 and then store it into location1. In the finally block, it loads string “changed in finally” and store it to location0. And at the end, it loads the value from location1 onto stack for return value and exits the method.

So we can simply put, the string “original string” is prepared for return before the finally block gets executed, that is why the “original string” was returned from the method.

.method private hidebysig static string  GetString() cil managed

{

  // Code size       24 (0x18)

  .maxstack  1

  .locals init (string V_0,

           string V_1)

  IL_0000:  nop

  IL_0001:  ldstr      "original string"

  IL_0006:  stloc.0

  .try

  {

    IL_0007:  nop

    IL_0008:  ldloc.0

    IL_0009:  stloc.1

    IL_000a:  leave.s    IL_0015

  }  // end .try

  finally

  {

    IL_000c:  nop

    IL_000d:  ldstr      "changed in finally"

    IL_0012:  stloc.0

    IL_0013:  nop

    IL_0014:  endfinally

  }  // end handler

  IL_0015:  nop

  IL_0016:  ldloc.1

  IL_0017:  ret

} // end of method SimpleTest::GetString

Posted On Thursday, November 27, 2008 9:01 PM | Comments (12)

Saturday, October 4, 2008 #

Interesting books

Some interesting books are going to be released later this year and can be pre-ordered from Amazon now.


Concurrent Programming on Windows by Joe Duffy Nov 2008


Framework Design Guidelines: Conventions, Idioms, and Patterns for Reusable .NET Libraries (2nd Edition) by Krzysztof Cwalina and Brad Abrams Nov 2008


Windows® Internals: Including Windows Server 2008 and Windows Vista, Fifth Edition by Mark Russinovich and David A. Solomon Nov 2008


Programming Entity Framework by Julia Lerman Jan 2009


Data Services with Silverlight 2 by John Papa Dec 2008

Posted On Saturday, October 4, 2008 9:22 PM | Comments (1)

Friday, October 3, 2008 #

How many projects should be in a Visual Studio solution?

Jeremy D. Miller, kicked off a good discussion on how many projects should be in a Visual studio solution with his latest post Separate Assemblies != Loose Coupling.

There are also a few more good posts on same topic:

·         Chad Myer’s Project Anti-Pattern: Many projects in a Visual Studio solution

·         Scott Hanselman’s Assembly Fiefdom: What’s the right number of assemblies / libraries?

·         Ayende’s The two project solution

·         Jacob Lewallen’s Projects in Visual Studio

I completely agree with Jeremy that we should keep number of assemblies minimum and do not separate until it is 100% necessary. Until not a long ago, I actually tended to have many projects in my solution. I think I was largely influenced by .NET Petshop 4.0 which I used for a reference project a few years ago. If you want to know what its project structure looks like. Here is it:

So, you can see, for such small application, there are 22 projects and a few of them just contain only one interface file. I would say this is a ridiculous overkill today, but to be honest, I kind of liked it a few years ago. It made me feel it was well designed/ structured/separated, and I can proudly tell my friends that I am working on a huge enterprise system which has  dozens of projects.

After worked with a couple of systems having a lot of projects, I have completely changed my mind. One system I worked with, it has 3 Visual Studio solutions for server, client and some sort of infrastructure level stuff. The sever solution has more than 80 projects, client has around 50 and infrastructure has more than 20. It wasn't me who created so many projects in the first place. I said I liked many projects, but I wouldn't go that far. Having so many projects slows down everything, if I accidentally kick off a “Rebuild”, I would have to go and get myself a cup of tea, have a chat with every colleague in the office, and come back. The time wasted, actually isn’t the biggest problem, more important problem is, developers get frustrated with it, and frustrated developers don’t enjoy working on it, they will not try to recompile immediately after they make a change, they will not do enough tests, they will not check in quickly. For example, if a developer is to add a new feature into the system, he will only rebuild the system and test it after he thinks he has completed the task, instead of breaking down the task into a number of smaller ones, and targeting one at the time.  Another important problem is, the project structure should provide some sort of high level overview of the system, i.e. how many components / layers in the system, how do they communicate to each other. Try to get a good picture of a system from a hundred projects is like to figure out how a car works from its nuts and bolts.

Another system I worked with has about 60 projects. Not as many as the first one, but it has more complicated dependencies among the projects. I couldn’t even rebuild whole solution from Visual Studio, because there is some wired circular reference, and of course I couldn’t run from Visual Studio too. What I had to do is to build the individual project I have made changes, copy all the assemblies to a particular location and run it from there, if I need debug through, I had to attach to the process from Visual Studio. So if anyone thinks separated assemblies enforce separation of concerns or decoupling, I could tell you, no, it won’t help you do that.

So, how many projects I will have in a Visual Studio solution? I think for a small application, something around 5 should be enough. For a more complicated system, 10 would be still reasonable. 20 projects probably is the maximum for both a developer and Visual Studio to manage effectively and efficiently, so, I would be very carful and reluctant to go over 20.

Posted On Friday, October 3, 2008 9:22 PM | Comments (1)

Sunday, September 21, 2008 #

Castle ActiveRecord with multiple databases

A few months ago, I came cross this problem that I had 2 databases in my project, so that my model classes might map to tables from different databases. It took me a while to work out how to deal with it in ActiveRecord. Today, I had a same problem again, even though I still got my last project’s source code, it still took me a couple of hours to get everything right. So, I think I’d better to write it done, just in case that I will have to do it again or anyone else may need it.  

The Scenario:

Database 1:      MyDatabase                             Table:  Product

Database 2:      MyProfileDatabase                   Table: UserProfile

Model Class:    Product, UserProfile

 The Code:

First, just define Product class as normal ActiveRecord model class:

[ActiveRecord]

public class Product : ActiveRecordBase<Product>

{

    [PrimaryKey("ProductID")]

    public string Id { get; set; }

   

    //...

}

 

Create an abstract base class for MyProfileDatabase.  All classes map to tables in MyProfileDatabase will inherit from this base class. Note: it has to be abstract. This was what I missed out and caused some fun time to find out.

 

public abstract class ProfileActiveRecord<T> : ActiveRecordBase<T>

{

}

 

Define UserProfile class which inherits from ProfileActiveRecord.

[ActiveRecord]

public class UserProfile : ProfileActiveRecord<UserProfile>

{

    [PrimaryKey("UserID")]

    public int Id { get; set; }

 

    //....

}

 

ActiveRecord Configration:

<activerecord>

  <config>

    <add key="hibernate.connection.driver_class"

         value="NHibernate.Driver.SqlClientDriver" />

    <add key="hibernate.dialect"

         value="NHibernate.Dialect.MsSql2005Dialect" />

    <add key="hibernate.connection.provider"

         value="NHibernate.Connection.DriverConnectionProvider" />

    <add key="hibernate.connection.connection_string"

         value="server=localhost;database=MyDatabase;Integrated Security=True" />

  </config>

 

  <config type="SomeNamespace.Core.Models.ProfileActiveRecord`1, SomeNamespace.Core">

    <add key="hibernate.connection.driver_class"

         value="NHibernate.Driver.SqlClientDriver" />

    <add key="hibernate.dialect"

         value="NHibernate.Dialect.MsSql2005Dialect" />

    <add key="hibernate.connection.provider"

         value="NHibernate.Connection.DriverConnectionProvider" />

    <add key="hibernate.connection.connection_string"

         value="server=localhost; database=MyProfileDatabase; Integrated Security=True" />

  </config>

</activerecord>

Initialize Active Records:

XmlConfigurationSource source = new XmlConfigurationSource("ActiveRecord.xml");

Type[] types =

    {

        typeof(Product),

        typeof(ProfileActiveRecord<>),

        typeof(UserProfile)

    };

ActiveRecordStarter.Initialize(source, types);

 

Posted On Sunday, September 21, 2008 7:07 AM | Comments (4)

Thursday, September 11, 2008 #

Debug into .Net Framework Source Code

I was writing some WPF testing code today. It was extremely simple code which just programmatically creates a Rectangle and adds it to a canvas. The code this something like this:

    Rectangle rectangle = new Rectangle

    {

        Fill = new SolidColorBrush(Colors.Blue),

        Stroke = new SolidColorBrush(Colors.Blue),

        Width = 150,

        Height = 120

    };

    rectangle.SetValue(Canvas.LeftProperty, 100);

    rectangle.SetValue(Canvas.TopProperty, 100);

    canvas.Children.Add(rectangle);

 

However, I got an exception on rectangle.SetValue(Canvas.LeftProperty, 100), saying "'100' is not a valid value for property 'Left'."

 

It didn’t take me long to release that Canvas.LeftProperty actually requires a double instead of an integer, but I was surprised that it couldn’t handle an integer. Normally, in a .NET program, we wouldn’t have to worry about the implicit cast from integer to double, right? So, I was just curious and want to find more about how DependencyObject.SetValue is implemented and wanted to have a look of the .NET framework code.

 

Normally, when I want to do this, I will just use .NET Reflector, but I just heard from my friends that I can debug into .NET framework code. So, why not give it a go?

 

So, followed ShawnBurke's great post Configuring Visual Studio to Debug .NET Framework Source Code, I got my Visual Studio set up and ready to go.

 

Now, fire up my program, set a break point, and in the CallStack panel, there are list of items are greyed out which are the .NET Framework code as Visual Studio doesn’t have the debug symbols. So, just right click, and "Load Symbols", Visual Studio will load them from the debug symbol server you have specified (ShawnBurke’s post above explains how to do it).

 

 

When the debug symbols loaded (it may take a few minutes. PresentationFramework.pdb has 12Mb and PresentationCore.pdb has 16Mb), you will know as greyed out items on CallStack turn black.

Hit F11, now, you can see this:

 

 

 

Cool, isn’t it? We are now in .NET framework source code! Writing a .NET program could be as cool as writing in java! (just kidding)

 

After I played around a bit and dug through some .NET framework code, I found it pretty much just check the value passed in is an instance of the type DependencyProperty defined of. The code is something like this:

 

    if (!propertyType.IsInstanceOfType(value))

    {

        return false;

    }

 

If it returns false, an exception will be thrown from SetValueCommon.I guess WPF could have done better to handle an integer as a double, after all, if we want to set a location programmatically, normally we would just use an integer, right? Anyway, it is not really my point for this post. I really just want to show how to debug into .NET framework code here.

 

Pretty cool, isn’t it? So, what is not so cool?

Well, did find a couple of things not as cool as I expected.

 

  • You have to load debug symbols every time you run the program. It will load from your local cache once it is downloaded from the server, so it won’t take long. But you do have to right click the CallStack, and "Load Symbols" every time.
  • In .Net Framework code, variable values will not be shown properly. You will have "Cannot evaluate expression because the code of the current method is optimized."

 

Posted On Thursday, September 11, 2008 6:53 AM | Comments (1)

Wednesday, September 10, 2008 #

Thoughts on programming language and maintainability

Recently, I have been working on maintaining two “legacy” systems. One is in VB6, implemented by a person who was not a programmer, but only trained a few weeks for VB6. The code quality is extremely poor. There are even no indentations in the code, and all variables are not properly named, for example, if there are 5 text boxes on a form, they will be just named as Text1(0), Text1(1)…Text(4).  So, I guess I don’t have to say too much and you get a picture of what it looks like. Another system is done with .NET 1.1 using C#, and it is written primarily by a guy who, is considered as a coding hero or maybe, more precisely, a coding cowboy. I didn’t work with any VB stuff before this one, and I have been using C# for 4-5 years.

So, how easy I found maintaining these two systems? This probably will surprise you, but I do find maintaining the VB system is much easier.

Why I find maintaining the VB system is easier? Let me tell you what I normally do to fix a bug. Normally, I will start from finding the form. All the form files are inside one folder, and thank god, they are named properly. So it is not a big deal to find it. And then I will find the control, normally a button which triggers off the function which has a problem. I will then double click on the button, and it will take me to the event handler in the code view. There are normally at least a few hundred lines of spaghetti code. I do sometimes find it is a bit frustrating to work on some code like this, but I will tell myself to be patient, and normally it won’t take too long to locate the line has problem, and fix it normally not a big problem, and I won’t have to worry about fix this one will cause another ten bugs elsewhere because there is no code sharing at all.

While, fixing a bug in the .NET system is completely a different story. Normally finding a file is a big challenge. There are more than 50 projects in the solution (not because the system is large, it could be just 5 – 10), a project may contain only one file, but there are 10000 lines code in it and dozens of class definitions. There are interfaces which you will never find what class implemented it, and there are stuff may be loaded into system via reflection on runtime. There might be some super complicated and wired inheritance structure. Sometimes I get impression, that it is designed not to be found.  I don’t want list of the problems here, I guess everyone has worked on this kind of system knows what I am talking about.

The point I want to raise here is, in term of maintainability, maintaining a well implemented object-oriented system might be easier than a well designed procedural system, but maintaining an object-oriented crap is even more difficult than doing a procedural crap. Also, no architecture is generally better than over-architecture. I think there are some reasons:

  • An object-oriented language itself is more complicated than a procedural language. The cool features like inheritance and polymorphism are all adding complicity and indirection into the program. Note, complicity and indirection are very bad thing, so if they don’t buy you anything, they should not be there.
  • A procedural language might not express well (spaghetti code), but it doesn’t lie, but an object-oriented language does. It could deceive you by using cool stuff like inheritance, reflection, delegate, extension methods, etc.

So, don’t take it for granted that an object-oriented language will automatically make your system more maintainable. What does make an object-oriented language work is the “loose coupling, high cohesion”, is single responsibility principle, is the open-close principle. It is not the cool language features or cool design patterns.

    

Posted On Wednesday, September 10, 2008 7:25 AM | Comments (2)

Monday, September 8, 2008 #

Turn off Resharper’s suggestion of using implicitly typed locals

If you use resharper with Visual Studio 2008, you may have noticed that it keeps suggesting you use implicitly typed local variable declaration i.e. the var keyword. To me, it is a bit annoying.

 

You can actually find why people think it should exist in the first place:http://resharper.blogspot.com/2008/03/varification-using-implicitly-typed.html

 

There are some valid points, but I am not convinced. I think it is less readable in most of the case. I can’t agree with Ilya that it induces better naming for local variables or better API. You got to name your variables properly no matter what you use, right? It won't make a good programmer better, but it will make a bad programmer even worse.

 

I do agree that compare to

Dictionary<string, Employee> employees = new Dictionary<string, Employee>();

That var employees = new Dictionary<string, Employee>();

does have same readability and it does save you a few key strikes. The benefit is really small, and you have to make judgment on it. The problem is you may have different ideas at different time and different team members may have their own habit. So, in your team, if the coding standard is “Use var whenever you think it good”, it will inevitably cause inconstant coding style. I would rather avoid it by having “Never use var”, you would lost too much after all.

 

So, if you don't like it neither, this is the way to turn it off. From “Resharper” menu, go to “Options…”, and from “Code Inspection”, then from “Inspection Severity”, find "Use Var keyword when possible” and set to “Do not show”

 

 

Posted On Monday, September 8, 2008 6:24 AM | Comments (13)

Copyright © Changhong Fu

Design by Bartosz Brzezinski

Design by Phil Haack Based On A Design By Bartosz Brzezinski