Category Archives: .NET

技术整理

技术站点

  • Hacker News:非常棒的针对编程的链接聚合网站
  • Programming reddit:同上
  • MSDN:微软相关的官方技术集中地,主要是文档类
  • infoq:企业级应用,关注软件开发领域
  • OSChina:开源技术社区,开源方面做的不错哦
  • cnblogs,51cto,csdn:常见的技术社区,各有专长
  • stackoverflow:IT技术问答网站
  • GitHub:全球最大的源代码管理平台,很多知名开源项目都在上面,如Linux内核,
  • OpenStack等免费的it电子书:http://it-ebooks.info/
  • DevStore:开发者服务商店

不错的书籍

  • 人件
  • 人月神话
  • 代码大全2
  • 计算机程序设计艺术
  • 程序员的自我修养
  • 程序员修炼之道
  • 高效能程序员的修炼(成为一名杰出的程序员其实跟写代码没有太大关系)
  • 深入理解计算机系统
  • 软件随想录
  • 算法导论(麻省理工学院出版社)
  • 离线数学及其应用
  • 设计模式
  • 编程之美
  • 黑客与画家
  • 编程珠玑
  • C++ Prime
  • Effective C++
  • TCP/IP详解
  • Unix 编程艺术
  • 《精神分析引论》弗洛伊德
  • 搞定:无压力工作的艺术

平台工具(都是开源的好东东哦)

  • Redmine/Trac:项目管理平台
  • Jenkins/Jira(非开源):持续集成系统(Apache Continuum,这个是Apache下的CI系统,还没来得及研究)
  • Sonar:代码质量管理平台
  • git,svn:源代码版本控制系统
  • GitLib/Gitorious:构建自己的GitHub服务器
  • gitbook:https://www.gitbook.io/写书的好东西,当然用来写文档也很不错的
  • Travis-ci:开源项目持续集成必备,和GitHub相结合,https://travis-ci.org/
  • 开源测试工具、社区(Selenium、OpenQA.org)
  • Puppet:一个自动管理引擎,可以适用于Linux、Unix以及Windows平台。所谓配置管理系统,就是管理机器里面诸如文件、用户、进程、软件包这些资源。无论是管理1台,还是上万台机器Puppet都能轻松搞定。
  • Nagios:系统状态监控报警,还有个Icinga(完全兼容nagios所有的插件,工作原理,配置文件以及方法,几乎一模一样。配置简单,功能强大)
  • Ganglia:分布式监控系统
  • fleet:分布式init系统

爬虫相关(好玩的工具)

  • Phantomjs
  • berserkJS(基于Phantomjs的改进版本)
  • SlimerJS
  • CasperJS
  • selenium

Web 服务器性能/压力测试工具/负载均衡器

  • http_load: 程序非常小,解压后也不到100K
  • webbench: 是Linux下的一个网站压力测试工具,最多可以模拟3万个并发连接去测试网站的负载能力
  • ab: ab是apache自带的一款功能强大的测试工具
  • Siege: 一款开源的压力测试工具,可以根据配置对一个WEB站点进行多用户的并发访问,记录每个用户所有请求过程的相应时间,并在一定数量的并发访问下重复进行。
  • squid(前端缓存),nginx(负载),nodejs(没错它也可以,自己写点代码就能实现高性能的负载均衡器):常用的负载均衡器
  • Piwik:开源网站访问量统计系统
  • ClickHeat:开源的网站点击情况热力图
  • HAProxy:高性能TCP /HTTP负载均衡器
  • ElasticSearch:搜索引擎基于Lucene
  • Page Speed SDK和YSLOW
  • HAR Viewer: HAR分析工具
  • protractor:E2E(end to end)自动化测试工具

Web 前端相关

  • GRUNT: js task runner
  • Sea.js: js模块化
  • knockout.js:MVVM开发前台,绑定技术
  • Angular.js: 使用超动感HTML & JS开发WEB应用!
  • Highcharts.js,Flot:常用的Web图表插件
  • Raw:非常不错的一款高级数据可视化工具
  • Rickshaw:时序图标库,可用于构建实时图表
  • JavaScript InfoVis Toolkit:另一款Web数据可视化插件
  • Pdf.js,在html中展现pdf
  • ACE,CodeMirror:Html代码编辑器(ACE甚好啊)
  • NProcess:绚丽的加载进度条
  • impress.js:让你制作出令人眩目的内容展示效果(类似的还有reveal)
  • Threejs:3DWeb库
  • Hightopo:基于Html5的2D、3D可视化UI库
  • jQuery.dataTables.js:高度灵活的表格插件
  • Raphaël:js,canvas绘图库,后来发现百度指数的图形就是用它绘出来的
  • director.js:js路由模块,前端路由,Nodejs后端路由等,适合构造单页应用
  • pace.js:页面加载进度条
  • bower:Web包管理器
  • jsnice:有趣的js反编译工具,猜压缩后的变量名 http://www.jsnice.org/
  • D3.js: 是一个基于JavaScript数据展示库(类似的还有P5.js)
  • Zepto.js:移动端替代jQuery的东东,当然也可以使用jquery-mobile.

UI框架:Foundation,Boostrap,Pure,EasyUI,Polymer

前端UI设计师必去的几个网站:Dribbble,awwwards,unmatchedstyle,UIMaker

Mozilla 开发者中心:https://developer.mozilla.org/en-US/

图标资源:IcoMoon(我的最爱),Themify Icons,FreePik,Glyphiconsart

  • Dialog:非常漂亮的对话框
  • AdminLTE:github上的一个开源项目,基于Boostrap3的后台管理页面框架
  • Respond.js:让不懂爱的IE6-8支持响应式设计
  • require.js: js模块加载库
  • select2:比chosen具有更多特性的选择框替代库
  • AngularUI:集成angular.js的UI库
  • normalize.css: 采用了现代化标准让各浏览器渲染出的html保持一致的库
  • CreateJS:Html5游戏引擎Less,Compass:简化CSS开发
  • emojify.js:用于自动识别网页上的Emoji文字并将其显示为图像
  • simditor:一个不错的开源的html编辑器,简洁高效
  • Sencha: 基于html5的移动端开发框架
  • SuperScrollorama+TweenMax+skrollr:打造超酷的视差滚动效果网页动画
  • jquery-smooth-scroll:同上,平滑滚动插件
  • Animate.css:实现了各种动画效果的css库
  • Emmet:前端工程师必备,ZenCode的前身
  • MagicDraw:Uml图工具

大数据处理/数据分析/分布式工具

  • Hadoop:分布式的文件系统,结合其MapReduce编程模型可以用来做海量数据的批处理(Hive,Pig,HBase啥的就不说了),值得介绍的是Cloudera的Hadoop分支CDH5,基于YARN MRv2集成了Spark可直接用于生产环境的Hadoop,对于企业快速构建数据仓库非常有用。
  • Ceph:Linux分布式文件系统(特点:无中心)
  • Storm:实时流数据处理,可以看下IBM的一篇介绍 (还有个Yahoo的S4,也是做流数据处理的)
  • Spark:大规模流式数据处理(可以应付企业中常见的三种数据处理场景:复杂的批量数据处理(batch data processing);基于历史数据的交互式查询(interactive query);基于实时数据流的数据处理(streaming data processing)),CSND有篇文章介绍的不错
  • Spark Streaming:基于Spark的实时计算框架
  • Tachyon:分布式内存文件系统
  • Mesos:计算框架一个集群管理器,提供了有效的、跨分布式应用或框架的资源隔离和共享Impala:新一代开源大数据分析引擎,提供Sql语义,比- Hive强在速度上
  • SNAPPY:快速的数据压缩系统,适用于Hadoop生态系统中
  • Kafka:高吞吐量的分布式消息队列系统
  • ActiveMQ:是Apache出品,最流行的,能力强劲的开源消息总线
  • MQTT:Message Queuing Telemetry Transport,消息队列遥测传输)是IBM开发的一个即时通讯协议,有可能成为物联网的重要组成部分
  • RabbitMQ:记得OpenStack就是用的这个东西吧
  • ZeroMQ:宣称是将分布式计算变得更简单,是个分布式消息队列,可以看下云风的一篇文章的介绍开源的日志收集系统:scribe、chukwa、kafka、flume。这有一篇对比文章
  • Zookeeper:可靠的分布式协调的开源项目
  • Databus:LinkedIn 实时低延迟数据抓取系统

数据源获取:Flume、Google Refine、Needlebase、ScraperWiki、BloomReach

序列化技术:JSON、BSON、Thrift、Avro、Google Protocol Buffers

NoSql:Apache Hadoop、Apache Casandra、MongoDB、Apache CouchDB、Redis、BigTable、HBase、Hypertable、Voldemort、Neo4j

MapReduce相关:Hive、Pig、Cascading、Cascalog、mrjob、Caffeine、S4、MapR、Acunu、Flume、Kafka、Azkaban、Oozie、Greenplum

数据处理:R、Yahoo! Pipes、Mechanical Turk、Solr/ Lucene、ElasticSearch、Datameer、Bigsheets、TinkerpopNLP自然语言处理:Natural Language Toolkit、Apache OpenNLP、Boilerpipe、OpenCalais

机器学习:WEKA、Mahout、scikits.learn、SkyTree

可视化技术:GraphViz、Processing、Protovis、Google Fusion Tables、Tableau、Highcharts、EChats(百度的还不错)、Raphaël.js

  • Kettle:开源的ETL工具
  • Pentaho:以工作流为核心的开源BI系统
  • Mondrian:开源的Rolap服务器
  • Oozie:开源hadoop的工作流调度引擎

开源的数据分析可视化工具:Weka、Orange、KNIME

Cobar:阿里巴巴的MySql分布式中间件

C & C++

Thrift:用来进行可扩展且跨语言的服务的开发(类似的还有个Avro,Google protobuf)。

libevent:是一个事件触发的网络库,适用于windows、linux、bsd等多种平台,内部使用select、epoll、kqueue等系统调用管理事件机制。(对了还有个libev呢)

Boost:不多说了,准C++标准库

Ptmalloc\Valgrind\Purify

NetworkServer架构:acceptor->dispatcher->worker(这个不算工具哦)

breakpad:崩溃转储和分析模块,很多crashreport会用到

UI界面相关:MFC、BCG和QT这类的就不说了,高端一点的还有Html和DirectUI技术:libcef(基于chrome内核的,想想使用html5开发页面,还真有点小激动呢)、HtmlLayout、Duilib、Bolt,非C++的,还有node-webkit也不错,集成了node和webkit内核。

游戏开发相关

MINA:使用Java开发手游和页游服务器(对了还有Netty,也很猛的,都是基于NIO的)

HP-Socket:见有有些页游服务器使用这个构建的

云风的技术博客:http://blog.codingnow.com/

OGRE:大名鼎鼎的3D图形渲染引擎

OpenVDB:梦工厂C++的特效库,开源的

cocos2d:跨平台2D游戏引擎

unity3d:跨平台3D游戏引擎,很火的哦

Nodejs:也有不少使用它来开发手游和也有服务器(网易的Pomelo就是哦)

日志聚合,分布式日志收集

Scribe:Facebook的(nodejs + scribe + inotify 同步日志)

logstash:强大的日志收集系统,可以基于logstash+kibana+elasticsearch+redis开发强大的日志分析平台

log.io: nodejs开发的实时日志收集系统

RTP,实时传输协议与音视频

RTP,RTCP,RTSP-> librtp,JRTPLIB(遵循了RFC1889标准)

环形缓冲区,实时数据传输用

SDL,ffmpeg,live555,Speex

Red5:用Java开发开源的Flash流媒体服务器。它支持:把音频(MP3)和视频(FLV)转换成播放流; 录制客户端播放流(只支持FLV);共享对象;现场直播流发布;远程调用。

Python

Eric,Eclipse+pydev,比较不错的Python IDE

PyWin:Win32 api编程包

numpy:科学计算包,主要用来处理大型矩阵计算等,此外还有SciPy,Matplotlib

GUI相关:PyQt,PyQwt

supervisor:进程监控工具

Java相关

常用的IDE:IntelliJ IDEA,Eclipse,Netbeans

Web开发相关:Tomcat、Resin、Jetty、WebLogic等,常用的组件Struts,Spring

HibernateNetty: 异步事件驱动网络应用编程框架,用于高并发网络编程比较好(NIO框架)

MINA:简单地开发高性能和高可靠性的网络应用程序(也是个NIO框架),不少手游服务端是用它开发的

jOOQ:java Orm框架Activiti:工作流引擎,类似的还有jBPM、Snaker

Perfuse:是一个用户界面包用来把有结构与无结构数据以具有交互性的可视化图形展示出来.

Gephi:复杂网络分析软件, 其主要用于各种网络和复杂系统,动态和分层图的交互可视化与探测开源工具

Nutch:知名的爬虫项目,hadoop就是从这个项目中发展出来的

web-harvest:Web数据提取工具

POM工具:Maven+ArtifactoryNetflix

Curator:Netflix公司开源的一个Zookeeper client library,用于简化Zookeeper客户端编程

Akka:一款基于actor模型实现的 并发处理框架

EclEmma:覆盖测试工具

.net相关

Xilium.CefGlue:基于CEF框架的.NET封装,基于.NET开发Chrome内核浏览器

CefSharp:同上,有一款WebKit的封装,C#和Js交互会更简单

netz:免费的 .NET 可执行文件压缩工具

SmartAssembly:变态的.net代码优化混淆工具

NETDeob0:.net反混淆工具,真是魔高一尺道高一丈啊(还有个de4dot,在GitHub上,都是开源的)

ILMerge:将所有引用的DLL和exe文件打成一个exe文件

ILSpy:开源.net程序反编译工具

Javascript.NET:很不错的js执行引擎,对v8做了封装

NPOI: Excel操作

DotRAS:远程访问服务的模块

WinHtmlEditor: Winform下的html编辑器

SmartThreadPool:使用C#实现的,带高级特性的线程池

Snoop: WPF Spy Utility

Autofac: 轻量级IoC框架

HtmlAgilityPack:Html解析利器

Quartz.NET:Job调度

HttpLib:@CodePlex,简化http请求

SuperSocket:简化Socket操作,基于他的还有个SuperWebSocket,可以开发独立的WebSocket服务器了

DocX:未安装Office的情况下操作Word文件

Dapper:轻量级的ORM类,性能不错

HubbleDotNet:支持接入数据库的全文搜索系统

fastJSON:@CodeProject,高性能的json序列化类

ZXing.NET:@CodePlex,QR,条形码相关

Nancy:轻量级Http服务器,做个小型的Web应用可以摆脱IIS喽(Nancy.Viewengines.Razor,可以加入Razor引擎)

AntiXSS:微软的XSS防御库Microsoft Web Protection

LibraryJint:JavaScript解释器

CS-Script:将C#代码文件作为脚本执行

Jexus:Linux下 高性能、易用、免费的ASP.NET服务器

Clay:将dynamic发挥的更加灵活,像写js一样写C#

DynamicJSON:不必定义数据模型获取json数据

Antlr:开源的语法分析器(归到C#不太合适,其他语言也可以去用)

SharpPcap:C#版的WinPcap调用端,牛逼的网络包分析库(自带PacketNotNet用于包协议分析)

Roslyn:C#,VB编译器

ImageResizer: 服务端自由控制图片大小,真乃神器也,对手机端传小图,PC端传大图,CMS用它很方便

UI相关:DevExpress, Fluent(Office 07风格), mui(Modern UI for WPF)

NetSparkle:应用自动更新组件

ConfuserEx: 开源.net混淆工具

ServiceStack: 开源高性能Web服务框架,可用于构建高性能的REST服务Expression

Evaluator:Eval for C#,处理字符串表达式

http://nugetmusthaves.com/

常用工具

  • Fiddler:非常好用的Web前端调试工具,当然是针对底层http协议的,一般情况使用Chrome等自带的调试工具也足够了,特殊情况还得用它去处理
  • wireshark:知名的网络数据包分析工具
  • PowerCmd:替代Windows Cmd的利器
  • RegexBuddy:强大的正则表达式测试工具
  • Soure Insight:源代码阅读神器
  • SublimeText:程序员最爱的编辑器
  • Database.NET:一个通用的关系型数据库客户端,基于.NET 4.0开发的,做简单的处理还是蛮方便的
  • Navicat Premium:支持MySql、PostgreSQL、Oracle、Sqlite和SQL Server的客户端,通用性上不如Database.NET,但性能方面比Database.NET好很多,自带备份功能也用于数据库定时备份。
  • Synergy : 局域网内一套键盘鼠标控制多台电脑
  • DameWare:远程协助工具集(我在公司主要控制大屏幕用)
  • Radmin: 远程控制工具,用了一段时间的
  • DameWare,还要破解,对Win7支持的不好,还是发现这个好用
  • Listary:能极大幅度提高你 Windows 文件浏览与搜索速度效率的「超级神器」
  • Clover:给资源管理器加上多标签
  • WinLaunch:模拟Mac OS的Launch工具
  • Fritzing:绘制电路图
  • LICEcap:gif教程制作git,
  • svn:版本控制系统Enigma Virtual Box(将exe,dll等封装成一个可执行程序)
  • Open DBDiff(针对SqlServer)数据库同步
  • SymmetricDS:数据库同步
  • BIEE,Infomatica,SPSS,weka,R语言:数据分析
  • CodeSmith,LightSwitch:代码生成
  • Pandoc:Markdown转换工具,出书用的。以前玩过docbook,不过现在还是Markdown盛行啊。
  • Window Magnet[Mac]:增强Mac窗口管理功能,想Win7一样具有窗口拖放到屏幕边缘自动调整的功能
  • log explorer:查看SqlServer日志dependency
  • walker:查询Windows应用程序dll依赖项
  • Shairport4w:将iPhone,iPad,iPod上的音频通过AirPlay协议传输到PC上
  • ngrok:内网穿透工具Axure:快速原型制作工具,还有个在线作图的工具国内的一个创业团队做的,用着很不错 http://www.processon.com
  • tinyproxy:(Linux)小型的代理服务器支持http和https协议EaseUS Partition
  • Master:超级简单的分区调整工具,速度还是蛮快的,C盘不够用了就用它从D盘划点空间吧,不用重装系统这么折腾哦。
  • CheatEngine:玩游戏修改内存值必备神器(记得我在玩轩辕剑6的时候就用的它,超级方便呢)
  • ApkIDE:Android反编译神器翻、墙工具(自|由|门、天行浏览器)

设计工具:Sketch、OmniGraffle

MindManger:思维导图

from:https://segmentfault.com/q/1010000002404545

HTTP Request flow in IIS (Image)

Overview of an HTTP Request :

iis-1

The following list describes the request-processing flow :

  1. When a client browser initiates an HTTP request for a resource on the Web server, HTTP.sys intercepts the request.
  2. HTTP.sys contacts WAS to obtain information from the configuration store.
  3. WAS requests configuration information from the configuration store, applicationHost.config.
  4. The WWW Service receives configuration information, such as application pool and site configuration.
  5. The WWW Service uses the configuration information to configure HTTP.sys.
  6. WAS starts a worker process for the application pool to which the request was made.
  7. The worker process processes the request and returns a response to HTTP.sys.
  8. The client receives a response.

Detail of a HTTP request inside the Worker Process

iis-2

ASP.NET Request Flow:

iis-3

  • IIS gets the request
  • Looks up a script map extension and maps to aspnet_isapi.dll
  • Code hits the worker process (aspnet_wp.exe in IIS5 or w3wp.exe in IIS6)
  • .NET runtime is loaded
  • IsapiRuntime.ProcessRequest() called by non-managed code
  • IsapiWorkerRequest created once per request
  • HttpRuntime.ProcessRequest() called with Worker Request
  • HttpContext Object created by passing Worker Request as input
  • HttpApplication.GetApplicationInstance() called with Context to retrieve instance from pool
  • HttpApplication.Init() called to start pipeline event sequence and hook up modules and handlers
  • HttpApplicaton.ProcessRequest called to start processing
  • Pipeline events fire
  • Handlers are called and ProcessRequest method are fired
  • Control returns to pipeline and post request events fire

refer:

A low-level Look at the ASP.NET Architecture

IIS Architecture

Dynamic Type Using Reflection.Emit

Table of Contents

Hi Folks, Its long since I am writing this for you. I have been writing technical blogs though for you but really wanted to share one tutorial which would help the database. Finally thought of starting one.

 

After putting my efforts with Reflection classes, I thought I could make some research on code generation. I took the CodeDom being the best alternative to generate code. Sooner or later, I found out, CodeDom actually allows you to build your assembly but it is does not allow you to dynamically compile a part of the assembly at runtime, but rather it invokes the compiler to do that. So rather than CodeDom, I thought there must be something else which fruits my needs.

Next I found out one, using Expression Trees. If you are already following me, I think you know, few days back I have already written about Expression Trees and Lamda Decomposition. So it is not a good time to recap the same. Later on, I did some research on MSIL, and found it worth learning. If you are going to grow with .NET, it would be your added advantage if you know about MSIL. Hence I started looking at the MSIL. Finally I found out a number of classes which might help you to build a Type dynamically. Let me share the entire thing with you.

Introduction

Reflection.Emit like CodeDom allows you to build your custom assembly and provides you a number of Builder classes which might be compiled during Runtime, and hence invoke DLR capabilities of C#. The library also exposes one ILGenerator which might be used later to produce the actual MSIL by putting efforts to emit Operation codes. So finally after you write your OpCodes correctly, you could easily able to compile the type dynamically during runtime. In this post, I would use ILDASM to see the IL generated from our own class, that I define, and later on I would try to build the same class dynamically.

What is Reflection?

If you are struck with Reflection, then you need to really gear yourself a bit to go further. Let me give a brief explanation of Reflection. Reflection is actually a technique to read a managed dll which are referenced or not being referenced from the application and invoke its types. In other words, it is a mechanism to discover the types and call its properties at runtime. Say for instance, you have an external dll which writes logger information and sends to the server. In that case, you have two options.

 

  • You refer to the assembly directly and call its methods.
  • You use Reflection to load the assembly and call it using interfaces.

If you want to build really a decoupled architecture for your application, something like which could be plugged in later in the application, it is always better to choose the 2nd option. Let me clarify a bit more, say you want your customer to download the logging dll from your server and plugin to the application when needed. Believe me, there is no other alternative than using Reflection. Reflection classes allows you to load an external assembly to your application and call its types at run time.

To know more try Reflection Overview.

What is Reflection.Emit?

Being a part of Reflection, Reflection.Emit namespace list you a number of classes which you can use to build your type. As I have already told you, Reflection.Emit actually provides you some Builder classes like AssemblyBuilder, ModuleBuilder, ConstructorBuilder, MethodBuilder, EventBuilder, PropertyBuilder etc. which allows you to build your IL dynamically during run time. The ILGenerator provides you the capabilities to generate your IL and place the same for a method. Generally, it is very rare that a developer need these capabilities to generate an assembly at runtime, but it is great to find these capabilities present in the framework.

 

Now lets see what is required to build an assembly.

Steps to generate an Assembly

Now lets jump back to identify the steps to create the assembly :

  1. Create an Assembly in an Application Domain.AssemblyBuilder will help you in that.
  2. Create a Module inside the Assembly
  3. Create a number of Type inside a Module
  4. Add Properties, Methods, Events etc inside the Type.
  5. Use ILGenerator to write inside the Properties, Methods etc.

Basically these are the common steps to create your own dynamically created Assembly.

structure.JPG

From the above figure, you can clear how the entire structure of an CLR assembly goes. The AppDomain is the root of the hierarchy which creates Assembly, then Module, and then Type. If you see the IL more thoroughly, you will understand, Delegate is also a class inherited from System.MultiCastDelegate and Struct is derived from System.ValueTypes. Each type might contain its members, and each member Method or Properties can have its OPCodes, Locals and Parameters. Locals define the local variables you define inside a method body and OpCodes are the instruction codes for the IL.

Steps to create Dynamic Assembly

Now lets go step by step with IL to build one dynamic assembly yourself.

STEP 1. Create an Assembly Dynamically

Hide   Copy Code

To create an assembly you need to have the name of the assembly to uniquely identify the assembly. I have used AssemblyName class to name the assembly for us. AppDomain is the place where the Assembly will be created. This is very important, as application mght have issues while calling cross domain objects. To make it more simplistic, rather than creating a new AppDomain, I am using CurrentDomain where the program is running. Finally I have created an object of AssemblyBuilder, which eventually builds up the Assembly with unique name aname. The AssemblyBuilderAccess specifies the accessibility of the assembly to us. As I did, if you use Run, that means the assembly could only be run dynamically using Reflection, it cannot be saved for future use. Browse through each of the values to see the output.

Note : If you have defined a custom attribute for the assembly, you can easily go for SetCustomAttriburte to add your custom attribute to the assembly.

Few features of AssemblyBuilder (For further reading)

AssemblyBuilder allows you to define a number of features like :

  • AddResourceFile : Allows you to specify a file to be added as resource to the assembly.
  • DefineUnmanagedResource / DefineResource : Adds one Unmanaged Resource for the assembly
  • EntryPoint/SetEntryPoint : A special subroutine / method to be defined which will be called automatically when the Assembly is invoked.
  • SetCustomAttribute : Lets you specify Attributes for the assembly.
  • DefineDynamicModule : Defines the Module for the assembly where the actual code will contain.

There are lot more flexibility while building the assembly. .NET put everything that we could have with the library and exposed methods to ensure we can do that from Reflection.Emit. You can try MSDN to read more about the methods it exposes.

STEP 2 : Create a Module

Module is a part of the object where the actual classes will remain. A module is container for all the classes we place therein. Let us create a module for us.

Hide   Copy Code

Thus the method is actually expecting a ModuleName, a unique module name and the Filename to which the assembly will be exported to.

Few useful methods

Module exposes few methods like :

  • DefineEnum : Lets you define an Enum, it returns back an EnumBuilder.
  • DefineType : Lets you to define a type / class.
  • DefineManifestResource : A dll contains a binary manifest. This method lets you define the manifest for you.
  • DefinePInvokeMethod : Allows you to define a PInvoke method (COM) for the assembly.

STEP 3 : Create a Type

This is the main thing. To create a class, structure, delegate etc you need to define a TypeBuilder. Now from here onwards I will look into the actual IL generated using ILDASM and then produce the same output for you.

Hide   Copy Code

The above GetType method has two overloads. As you can see the first one is simple one where I have just specified the name of the class and the ModuleBuilder and the method returns the TypeBuilder.

In the second overload, I have put an additional param array of string which defines each Generic type for the class. GenericTypeParameterBuilder allows you to define the GenericTypeParameter. Once you define the GenericTypeParameters and set its constraint attributes, you can the builder.

Few useful methods

Compared to classes, TypeBuilder allows you to define full fledged structure with all the options you have. Some of them are :

  • DefineField / DefineMethod / DefineProperties / DefineEvent : You might use these methods to generate class members.
  • DefineMethodOverride : Allows you to override an existing method when the Type is inherited from another base type
  • DefineConstructor / DefineDefaultConstructor : Specifies the constructor for the current type.
  • AddInterfaceImplementation : Allows you to implement the current type from another interface.

STEP 4 : Create Method

Method are the building block of any program. We will define a number of Methods to clear up the concepts on how easily you could build a method from IL. For the time being, lets we create a dynamic method using MethodBuilder.

Hide   Shrink   Copy Code

So the above methods will return you the MethodBuilder which lets you to define your IL code. You can see I have specified 3 overloads for this. The overloads allows you to put parameters and also to put Generic Type parameters for the methods.

Now, after building your type you need to create Locals (local variables) and use OpCode instructions.

Using ILGenerator to Emit OpCodes

To define your OpCodes you will need ILGenerator. ILGenerator allows you to Emit IL for your method body and hence lets you create the instruction set for the method that you are about to build. Let me first introduce some of the few instruction sets that helps you to add two integer variables passed into the method and return a floating value as a result.

Hide   Copy Code

If you minutely see the above code, the code actually creates an Assembly in CurrentDomain and having a dynamically created type MyClass in it. The class hence created will contain the method Sum in it.

The lines with ILGenerator.Emit actually emits the IL to the method body Sum. Every method must declare local stack element to run its data. In IL, we declare Local variables before calling any instruction codes. Just like this, I have used DeclareLocal to declare a float32 Local in the method. The DeclareLocal method actually returns a LocalBuilder which you might use as well to manipulate the index of this variable. After we declare all the locals, we first load argument list Ldarg_1 and Ldarg_2(as first argument is implicit object this). The Add_Ovf actually adds the two loaded arguments and pass it to the local variable Stloc_0 (which represents the top element of the Stack or the Local variable we created at Index 0). Next the Ldloc_0 pops the value and returns it back to the external world.

Now here is the most easiest sample of producing your own Type. Now let me go a bit further to build a more concrete Type

A more concrete example

As we have already built our own Type, its time to give you an example in building a more concrete Type. Before we demonstrate let me show you the code which we are going to create dynamically during runtime and call its method to get the output. Please note that, I have tried to simplify the code in an extent so that it helps you to understand the code better..

Hide   Shrink   Copy Code

In the above class, I have actually declared an Interface IBuilder which later I have implemented to produce a class Builder. The class contains few methods, events, properties etc. to allow you understand each and every flexibilities you have with it.

ILDASM1.JPG

After you see the code lets open ILDASM and see how the IL looks like the picture above. Basically it contains two type .class

  1. .class interface for IBuilder
  2. .class for Builder implementing IBuilder.

Other than that you will see another type for the Main method for my console application which you can omit for time being as we are going to concentrate on the Type.

To build a dynamic type, the most important thing that we have discussed already are Builder classes. The BCL exposes a number of Builder classes that enables us to generate MSIL code dynamically during runtime and hence you can compile the same to produce the output.

From the above figure I have put some of the most important Builder classes marked in Red. But ultimately, you need instructions to run for your application. To write your IL Expression, Reflection.Emit provides a class call ILGenerator. ILGenerator (marked in blue) enables you to write your IL for a method or property. OpCodes are Operation code that determined Computer instructions. So while writing your instructions you need to pass your OpCodes and generate the instruction set for the Method.

Now going further with our example, let me demonstrate the code one by one so that you could easily build your own Code generator.

Implement IBuilder for your Assembly

Lets move to the actual implementation of IBuilder interface. As per our discussion, IBuilder contains 4 members, Sum, Divide, Multiply & substract.

Hide   Copy Code

So, as I am new to IL, lets use ILDASM to see what exactly the IL looks like and hence we will try to implement the same for our assembly.

Hmm… After I build and Open ILDASM to disassemble my assembly the IL it produces looks like

Hide   Copy Code

Let me explain the IL a bit.

  • In MSIL any type is defined with .class, hence our Type IBuilder gets one .class for it.
  • interface, abstract keyword identifies the Type to be abstract and hence you cannot create object of the same.
  • Auto specifies the LPSTR(Long Pointer to string) interpreted automatically.
  • Ansi specifies the LPSTR(Long Pointer to String) interpreted as ANSI.
  • Methods in IBuilder is identified using .method keyword.
  • Because of being member of an interface, the methods appear as abstract virtual.
  • instance keyword specifies the object to be non-static
  • hidebysig specifies the method can be hidden by both name and signature. Any normal method you define gets this flexibility in .NET.
  • NewSlot makes the member to get a slot in vtable. (A vtable is the memory area for the whole object. So whenever an object is created, a vtable is created each for each object and any object created within it gets an entry of vtable. The first member being the hidden pointer can be used to find members of the vtable)
  • cil managed is used to determine that the method is implemented in managed environment.

Now as you now understand the IL generated by the Interface, its time to get through to create the Type IBuilder. Lets build the code for it :

Hide   Shrink   Copy Code

In the above code we first create the TypeBuilder from ModuleBuilder.DefineType. You should note, I have added the TypeAttributes in the same way as it was in MSIL. After we create the TypeBuilder, we can next add up the methods. The DefineMethod method helps in building the methods as we define the MethodAttributes correctly. CallingConvensions.HasThis will make the method as instance method.. We also need to mention the ReturnType and argument types specifically. In this case I have specified the ReturnType as System.Single(float) and arguments as integers for our code. It should be noted, we need to use SetImplementationFlags to specify the methods to be cil managed.

So as our IBuilder interface is ready, its time to build our actual Type.

Implementing the Builder Class

Hmm, now its time to go final workaround with this. First I will create the basic class with methods only required to implement the interface IBuilder, later on we will add the delegate, event, a new method, a static method etc.

To create the basic Builder class, the first thing that we need is a Constructor. But as our constructor also adds few lines to initialize properties FirstNum and SecondNum, let me define them first.

1. Implementing the Type

In IL most of the type is .class, as the whole body depends on the Type header, it is good to start with by building the Type signature. In terms of IL, as in ILDASM it looks like:

Hide   Copy Code

So basically the class extends System.Object (as for any class which does not inherit form other class) and in our case it implements IBuilder.

I guess, building the type using TypeBuilder should be easy for you, let me build it again for you :

Hide   Copy Code

So here in the code I have implemented the Builder from parentBuilder which is the Type object of IBuilder interface. If you focus on the code, you will see I have specified BeforeFieldInit for the type, which means you can call the static members without initializing the object. I have also implemented the Type from System.Object according to IL.

2. Implementing the Field & Properties

As our type is ready now, let us add some field and properties on the Type. As shown in the Builder Type we have two fields to store numeric values each of which is wrapped around using property wrappers. Lets see what exactly I am talking about :

Hide   Copy Code

So FirstNum and SecondNum are the two Properties that we need to declare for our type. Now if you look back for the IL implementation, it looks like :

Hide   Shrink   Copy Code

So considering the IL, property seems to be a wrapper for two methods one with get_PropertyName and another with set_PropertyName where get_PropertyName returns the value and set_PropertyName sets the value. So according to IL defination if you are going to implement the code it will look like :

Hide   Copy Code

Ohh, its huge…. .Yes, let me explain. First I have added a Field firstNum which is a numeric private variable. A FieldBuilder helps you to add a field into the IL. To define a property you need to first define the property itself and then you have to define two methods one for Getter and one for Setter such that the Getter returns System.Int32 and the Setter takes System.Int32 as argument.

The OpCodes provide the entire expression set. ldarg loads the argument and Ldfld and Stfld loads and sets the fields into the field fFirst.

A Nice thing to talk in this regard

So you must understand now, a property actually reserves the methods with get_Property and set_Property with the same signature. Say for instance you define a class with the following :

Hide   Copy Code

The class will not compile, as get_First and set_First is already reserved and the compiler throws warning doing this.

weird.JPG

Isn’t it interesting to know ?

3. Implementing the Constructor

If your class does not define a constructor in it, the C# compiler automatically puts in a default constructor for you. How it does? Actually for any class, the default constructor for System.Object is automatically inherited to the object and hence you do not need to define a default constructor in such case. It will be written in IL only when you define the default constructor explicitly.

In our case, I have explicit declaration of a parametrized constructor, as it is good to show you the code for that.

Hide   Copy Code

To do this let me quickly show you how the constructor looks like in terms of IL.

Hide   Copy Code

To build a constructor, we need to create object of ConstructorBuilder which you can get from DefineConstructor method of TypeBuilder. If you see the IL, you can see, the IL actually calls the constructor of System.Object first. This is needed, as any object internally inherited from Base System.Object.

The setFirstNum and set_SecondNum is called from the IL to set the values for FirstNum and SecondNum of the class.

Hide   Copy Code

The SpecialName in MethodAttribute for Constructor lets the method to be special to CLR. Hence the name of the method ctor makes it a constructor of the class.

To call the constructor of System.Object, we need to fetch the constructor of the object. I have used Reflection to get ConstructorInfo from Type and passed in to Call OpCode. We emit the code as specified in the IL and hence the constructor will be generated.

An Interesting thing to remember

One interesting thing to remember about Reflection.Emit is that, it internally sends a hidden object to every method it calls. This is the implicit object call which we identify as “this” in C# or “Me” in Vb. Thus when we call Ldarg_0 for OpCodes, we are actually mentioning to the implicit object passed into the constructor as first argument. So any parameter we specify starts with Index 1.

The only difference between the Constructor and a normal method is that, a Constructor does not return a value. In CLR, a method actually returns the top element in the stack immediately if OpCodes.Ret is received. So if your stack loads a value into stack before calling Ret, you will get “Invalid Program” exception when you create object of the type. So in such cases Nop should be invoked before calling Ret to consume a processing cycle.

Now that we have defined the constructor let me move forward to define the methods.

4. Implementing the Methods from IBuilder

Now as our constructor is ready, its time to implement the IBuilder object and define the Methods for us. As we are going through with code, I think it must be clear to you how to build your own custom objects. Lets try out the Divide method of IBuilder and implement the same for us.

The Divide method that we have declared earlier looks like

Hide   Copy Code

So basically from this method, you can understand how you could call an external member function from an object like what I have done using Console.WriteLine here, and also lets you understand how you could use try/Catch block during your Code generation. So, without wasting time, lets Open up ILDASM again and see how different the code looks like

Hide   Shrink   Copy Code

Now the implmentation states that the method loads 1st and 2nd argument and use Div operation to divide the values and Conv.r4 actually converts the result to float32 value. The Local stack element declared as float32 is used here and the converted result is pushed back to the stack again using stloc.1. If everything works fine, the applicaion passes its control to IL_0024 resulting the method to return the local stack value in 1st position.

So let us implement the same using Builder objects

Hide   Shrink   Copy Code

To open a Try block in IL, you need to use BeginExceptionBlock. Be sure to store the Label so that you could move to specific IL instruction code whenever required. Now Before starting the Catch block which is notified using BeginCatchBlock we need to Leave the Try block using OpCodes.Leave with the LabelName. This will ensure that the application maintains Scope.

You can see, we could have more than one catch block, and each of them will be identified by the Type passed into BeginCatchBlock. Hence, we just Load the string inside Catch block and call the Method WriteLine of Console to show the string. Finally, again before calling EndExceptionBlock, we leave the Try/catch block.

You should note, whenever you are about to call a method, you need to use a MethodInfo object.

Building a Delegate

As it is easy to create the other methods for your Type, lets move further to create a Delegate for you. It would be a good idea to show you how to build a Delegate for a class. Building a delegate differs from building other members. Lets say we need to declare a delegate for our Type as :

Hide   Copy Code

Now after seeing the one liners, dont think that the declaration of IL would be as simple as this. The IL looks like :

Hide   Copy Code

Oh my god, a delegate is actually defined as a nested Type (.class) inherited from System.MulticastDelegate inside the actual Type Builder. It also declares methods like Invoke, BeginEnvoke and EndEnvoke inside the new Type declaration. So lets Build the Type to help you in this even though I think you can create this yourself now.

Hide   Copy Code

Now moving further, I would recommend you to try out other methods too, you can try using EventBuilder class to build events, CustomAttributeBuilder to build your own user defined Custom Attributes etc.

Wrapping up the Whole thing

Now after we have implemented all the methods, let me check whether I have correctly produced the IL or not.

Hide   Copy Code

You should note, I have used dynamic to avoid unnecessary usage of Reflection classes again.

So it looks fine and after I compile I get output like

ilcode.JPG

So the IL generated works great for us. You can also save the IL as Assembly using

Hide   Copy Code

Dealing with Bad IL

While building your IL, you may often come accross a situation that your IL does not compile.

invalidprogram.JPG

Never worry, .NET comes with a free tool which gets installed with your Visual Studio will help you rescue from these scenarios. The common type of exception that takes place is “Common Language Runtime detected an invalid program”. The tool is named as PeVerify.exe.

Once you install the Visual Studio, you can open console and try invoking the following statement

peverify.exe <assemblypath>\yourassembly.dll After it compiles the assembly again, it will give you the actual exception that took place while building the Type. You can read more about PEVerify from : MSDN Reference for PEVerify [^]

References

There are quite a number of references over internet which might help you in this topic. Lets enumerate them for you

History

Initial Post – 26th October 2010

Conclusion

Its fun creating the article for you. Even I am excited to put this effort to write one article for you. Its all new to me as well, but I tried my level best to put as many as I can. I hope you would appreciate my post.

If you think I did any mistake, please let me know, as I am no masterpiece in this, so that we could make this article better. Thank you for reading the article.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

from:http://www.codeproject.com/Articles/121568/Dynamic-Type-Using-Reflection-Emit

Table Value Parameters in SQL Server 2008 and .NET (C#)

Problem

We recently upgraded our database to SQL Server 2008 and I want to update our data access C# code to use Table Value Parameters with our stored procedures. In a previous tip I saw that Table Value Parameters were used with a Data Warehousing example. Can you show me how to implement Table Value Parameters with my .NET Application to insert multiple records using one round-trip?

Solution

Table Value Parameters is a new feature for developers in SQL Server 2008. It allows you to pass read-only table variables into a stored procedure. In the past I have used a comma delimited string or XML to simulate this purpose. I would have to parse out the string into a temp table similar to this example.

Once you have the table parameter passed into the stored procedure you can leverage the Table Value Parameter just like any other table except you cannot modify the parameter as it’s read-only. Below we will apply a common example to bulk insert data using one round trip.


Create table and table type

The following is a sample table that we will use in this example to insert items.

In order to use table value parameters you have to create a table type. The table type is used to describe the structure of the table value parameter. This is similar to making a strong type.

Using TVP in T-SQL

Now that you have a table type you have to declare an instance to use it. The following is an quick example that declares an instance of the table type an populates it with some values. Later we will use C# to create a data table that will populate an instance of a table type.

Using TVP in Stored Procedure

In order to use table value parameters in a stored procedure you have to declare an instance of the table type and specify it as read-only. It is mandatory for the table value parameter to be read-only so you cannot modify the data inside of the table type variable inside the stored procedure.

Below we will insert the data in the table value parameter into the dbo.items table.

Table Value Parameters in .NET (C#)

The following code below generates a data table in C# and it includes four rows. A data table is a common data type used to simulate a table. This data table will be used as our table value parameter and will be used by the stored procedure created above. The data table is not the only type that can be used for table value parameters in C#. The DataReader and list types are also acceptable.

Now that we have our data table created we can move on to the data access code. The majority of the code listed below will be the same for the majority of your data access code. The only difference is we will specify that the input type is sql data type as structured and we will pass in our data table to the input parameter. The two lines you need to focus on are underlined in the code section below.

SQL Profiler

When you use this tip you can setup SQL Server Profiler to capture the .NET calls. You will see the following three calls. You should notice that they are very similar to the T-SQL calls above.

SQL Profiler

Next Steps
  • Check out my blogfor more on SQL Server
  • Click hereto download the sample code used in this tip.
  • Check out several linkson Table Value Parameters
  • Click herefor an TVP example for Data Warehouse
  • Click here for an ASP.NET example using Table Value Paramerters

from:http://www.mssqltips.com/sqlservertip/2112/table-value-parameters-in-sql-server-2008-and-net-c/

 

Drill Into .NET Framework Internals to See How the CLR Creates Runtime Objects

Hanu Kommalapati and Tom Christian
This article discusses:

  • SystemDomain, SharedDomain, and DefaultDomain
  • Object layout and other memory specifics
  • Method table layout
  • Method dispatching
This article uses the following technologies:        .NET Framework, C#
          Since the common language runtime (CLR) will be the premiere infrastructure for building applications in Windows® for some time to come, gaining a deep understanding of it will help you build efficient, industrial-strength applications. In this article, we’ll explore CLR internals, including object instance layout, method table layout, method dispatching, interface-based dispatching, and various data structures.
We’ll be using very simple code samples written in C#, so any implicit references to language syntax should default to C#. Some of the data structures and algorithms discussed will change for the Microsoft® .NET Framework 2.0, but the concepts should largely remain the same. We’ll use the Visual Studio® .NET 2003 Debugger and the debugger extension Son of Strike (SOS) to peek into the data structures we discuss in this article. SOS understands CLR internal data structures and dumps out useful information. See the “Son of Strike” sidebar for loading SOS.dll into the Visual Studio .NET 2003 debugger process. Throughout the article, we will describe classes that have corresponding implementations in the Shared Source CLI (SSCLI), which you can download from msdn.microsoft.com/net/sscli. Figure 1 will help you navigate the megabytes of code in the SSCLI while searching for the referenced structures.
Figure 1 SSCLI Reference
Item SSCLI Path
AppDomain sscliclrsrcvmappdomain.hpp
AppDomainStringLiteralMap sscliclrsrcvmstringliteralmap.h
BaseDomain sscliclrsrcvmappdomain.hpp
ClassLoader sscliclrsrcvmclsload.hpp
EEClass sscliclrsrcvmclass.h
FieldDescs sscliclrsrcvmfield.h
GCHeap sscliclrsrcvmgc.h
GlobalStringLiteralMap sscliclrsrcvmstringliteralmap.h
HandleTable sscliclrsrcvmhandletable.h
InterfaceVTableMapMgr sscliclrsrcvmappdomain.hpp
Large Object Heap sscliclrsrcvmgc.h
LayoutKind sscliclrsrcbclsystemruntimeinteropserviceslayoutkind.cs
LoaderHeaps sscliclrsrcincutilcode.h
MethodDescs sscliclrsrcvmmethod.hpp
MethodTables sscliclrsrcvmclass.h
OBJECTREF sscliclrsrcvmtypehandle.h
SecurityContext sscliclrsrcvmsecurity.h
SecurityDescriptor sscliclrsrcvmsecurity.h
SharedDomain sscliclrsrcvmappdomain.hpp
StructLayoutAttribute sscliclrsrcbclsystemruntimeinteropservicesattributes.cs
SyncTableEntry sscliclrsrcvmsyncblk.h
System namespace sscliclrsrcbclsystem
SystemDomain sscliclrsrcvmappdomain.hpp
TypeHandle sscliclrsrcvmtypehandle.h
A word of caution before we start—the information provided in this article is only valid for the .NET Framework 1.1 (it’s also mostly true for Shared Source CLI 1.0, with the most notable exceptions being some interop scenarios) when running on the x86 platform. This information will change for the .NET Framework 2.0, so please do not build software that relies on the constancy of these internal structures.
Domains Created by the CLR Bootstrap
Before the CLR executes the first line of the managed code, it creates three application domains. Two of these are opaque from within the managed code and are not even visible to CLR hosts. They can only be created through the CLR bootstrapping process facilitated by the shim—mscoree.dll and mscorwks.dll (or mscorsvr.dll for multiprocessor systems). As you can see in Figure 2, these are the System Domain and the Shared Domain, which are singletons. The third domain is the Default AppDomain, an instance of the AppDomain class that is the only named domain. For simple CLR hosts such as a console program, the default domain name is composed of the executable image name. Additional domains can be created from within managed code using the AppDomain.CreateDomain method or from unmanaged hosting code using the ICORRuntimeHost interface. Complicated hosts like ASP.NET create multiple domains based on the number of applications in a given Web site.

Figure 2 Domains Created by the CLR Bootstrap
System Domain
The SystemDomain is responsible for creating and initializing the SharedDomain and the default AppDomain. It loads the system library mscorlib.dll into SharedDomain. It also keeps process-wide string literals interned implicitly or explicitly.
String interning is an optimization feature that’s a little bit heavy-handed in the .NET Framework 1.1, as the CLR does not give assemblies the opportunity to opt out of the feature. Nonetheless, it saves memory by having only a single instance of the string for a given literal across all the application domains.
SystemDomain is also responsible for generating process-wide interface IDs, which are used in creating InterfaceVtableMaps in each AppDomain. SystemDomain keeps track of all the domains in the process and implements functionality for loading and unloading the AppDomains.
SharedDomain
All of the domain-neutral code is loaded into SharedDomain. Mscorlib, the system library, is needed by the user code in all the AppDomains. It is automatically loaded into SharedDomain. Fundamental types from the System namespace like Object, ValueType, Array, Enum, String, and Delegate get preloaded into this domain during the CLR bootstrapping process. User code can also be loaded into this domain, using LoaderOptimization attributes specified by the CLR hosting app while calling CorBindToRuntimeEx. Console programs can load code into SharedDomain by annotating the app’s Main method with a System.LoaderOptimizationAttribute. SharedDomain also manages an assembly map indexed by the base address, which acts as a lookup table for managing shared dependencies of assemblies being loaded into DefaultDomain and of other AppDomains created in managed code. DefaultDomain is where non-shared user code is loaded.
DefaultDomain
DefaultDomain is an instance of AppDomain within which application code is typically executed. While some applications require additional AppDomains to be created at runtime (such as apps that have plug-in architectures or apps doing a significant amount of run-time code generation), most applications create one domain during their lifetime. All code that executes in this domain is context-bound at the domain level. If an application has multiple AppDomains, any cross-domain access will occur through .NET Remoting proxies. Additional intra-domain context boundaries can be created using types inherited from System.ContextBoundObject. Each AppDomain has its own SecurityDescriptor, SecurityContext, and DefaultContext, as well as its own loader heaps (High-Frequency Heap, Low-Frequency Heap, and Stub Heap), Handle Tables (Handle Table, Large Object Heap Handle Table), Interface Vtable Map Manager, and Assembly Cache.
LoaderHeaps
LoaderHeaps are meant for loading various runtime CLR artifacts and optimization artifacts that live for the lifetime of the domain. These heaps grow by predictable chunks to minimize fragmentation. LoaderHeaps are different from the garbage collector (GC) Heap (or multiple heaps in case of a symmetric multiprocessor or SMP) in that the GC Heap hosts object instances while LoaderHeaps hold together the type system. Frequently accessed artifacts like MethodTables, MethodDescs, FieldDescs, and Interface Maps get allocated on a HighFrequencyHeap, while less frequently accessed data structures, such as EEClass and ClassLoader and its lookup tables, get allocated on a LowFrequencyHeap. The StubHeap hosts stubs that facilitate code access security (CAS), COM wrapper calls, and P/Invoke.
Having examined the domains and LoaderHeaps at a high level, we’ll now look at the physical details of these in the context of the simple app in Figure 3. We stopped the program execution at “mc.Method1();” and dumped the domain information using the SOS debugger extension command, DumpDomain (see the “Son of Strike” sidebar for SOS loading information).  Here is the edited output:

Figure 3 Sample1.exe
Our console program, Sample1.exe, is loaded into an AppDomain which has a name “Sample1.exe.” Mscorlib.dll is loaded into the SharedDomain but it is also listed against the SystemDomain as it is the core system library. A HighFrequencyHeap, LowFrequencyHeap, and StubHeap are allocated in each domain. The SystemDomain and the SharedDomain use the same ClassLoader, while the Default AppDomain uses its own.
The output does not show the reserved and committed sizes of the loader heaps. The HighFrequencyHeap initial reserve size is 32KB and its commit size is 4KB. LowFrequencyHeap and StubHeaps are initially reserved with 8KB and committed at 4KB. Also not shown in the SOS output is the InterfaceVtableMap heap. Each domain has a InterfaceVtableMap (referred to here as IVMap) that is created on its own LoaderHeap during the domain initialization phase. The IVMap heap is reserved at 4KB and is committed at 4KB initially. We’ll discuss the significance of IVMap while exploring type layout in subsequent sections.
Figure 2 shows the default Process Heap, JIT Code Heap, GC Heap (for small objects) and Large Object Heap (for objects with size 85000 or more bytes) to illustrate the semantic difference between these and the loader heaps. The just-in-time (JIT) compiler generates x86 instructions and stores them on the JIT Code Heap. GC Heap and Large Object are the garbage-collected heaps on which managed objects are instantiated.
Type Fundamentals
A type is the fundamental unit of programming in .NET. In C#, a type can be declared using the class, struct, and interface keywords. Most types are explicitly created by the programmer, however, in special interoperability cases and remote object invocation (.NET Remoting) scenarios, the .NET CLR implicitly generates types. These generated types include COM and Runtime Callable Wrappers and Transparent Proxies.
We’ll explore .NET type fundamentals by starting from a stack frame that contains an object reference (typically, the stack is one of the locations from which an object instance begins life). The code shown in Figure 4 contains a simple program with a console entry point that calls a static method. Method1 creates an instance of type SmallClass which contains a byte array used in demonstrating the creation of an object instance on a Large Object Heap. The code is trivial, but will serve for our discussion.
Figure 4 Large Objects and Small Objects
Figure 5 shows snapshot of a typical fastcall stack frame stopped at a breakpoint at the “return smallObj;” line inside the Create method. (Fastcall is the .NET calling convention which specifies that arguments to functions are to be passed in registers, when possible, with all other arguments passed on the stack right to left and popped later by the called function.) The value type local variable objSize is inlined within the stack frame. Reference type variables like smallObj are stored as a fixed size (a 4-byte DWORD) on the stack and contain the address of object instances allocated on the normal GC Heap. In traditional C++, this is an object pointer; in the managed world it’s an object reference. Nonetheless, it contains the address of an object instance. We’ll use the term ObjectInstance for the data structure located at the address pointed to by the object reference.

Figure 5 SimpleProgram Stack Frame and Heaps
The smallObj object instance on the normal GC Heap contains a Byte[] called _largeObj, whose size is 85000 bytes (note that the figure shows 85016 bytes, which is the actual storage size). The CLR treats objects with sizes greater than or equal to 85000 bytes differently than the smaller objects. Large objects are allocated on a Large Object Heap (LOH), while smaller objects are created on a normal GC Heap, which optimizes the object allocation and garbage collection. The LOH is not compacted, whereas the GC Heap is compacted whenever a GC collection occurs. Moreover, the LOH is only collected on full GC collections.
The ObjectInstance of smallObj contains the TypeHandle that points to the MethodTable of the corresponding type. There will be one MethodTable for each declared type and all the object instances of the same type will point to the same MethodTable. This will contain information about the kind of type (interface, abstract class, concrete class, COM Wrapper, and proxy), the number of interfaces implemented, the interface map for method dispatch, the number of slots in the method table, and a table of slots that point to the implementations.
One important data structure MethodTable points to is EEClass. The CLR class loader creates EEClass from the metadata before MethodTable is laid out. In Figure 4, SmallClass’s MethodTable points to its EEClass. These structures point to their modules and assemblies. MethodTable and EEClass are typically allocated on the domain-specific loader heaps. Byte[] is a special case; the MethodTable and the EEClass are allocated on the loader heaps of the SharedDomain. Loader heaps are AppDomain-specific and any data structures already mentioned here, once loaded, will not go away until an AppDomain is unloaded. Also, the default AppDomain can’t be unloaded and hence the code lives until the CLR is shut down.
ObjectInstance
As we mentioned, all instances of value types are either inlined on the thread stack or inlined on the GC Heap. All reference types are created on the GC Heap or LOH. Figure 6shows a typical object instance layout. An object can be referenced from stack-based local variables, handle tables in the interop or P/Invoke scenarios, from registers (the this pointer and method arguments while executing a method), or from the finalizer queue for objects having finalizer methods. The OBJECTREF does not point to the beginning of the Object Instance but at a DWORD offset (4 bytes). The DWORD is called Object Header and holds an index (a 1-based syncblk number) into a SyncTableEntry table. As the chaining is through an index, the CLR can move the table around in memory while increasing the size as needed. The SyncTableEntry maintains a weak reference back to the object so that the SyncBlock ownership can be tracked by the CLR. Weak references enable the GC to collect the object when no other strong references exist. SyncTableEntry also stores a pointer to SyncBlock that contains useful information, but is rarely needed by all instances of an object. This information includes the object’s lock, its hash code, any thunking data, and its AppDomain index. For most object instances, there will be no storage allocated for the actual SyncBlock and the syncblk number will be zero. This will change when the execution thread hits statements like lock(obj) or obj.GetHashCode, as shown here:

Figure 6 Object Instance Layout
In this code, smallObj will use zero (no syncblk) as its starting syncblk number. The lock statement causes the CLR to create a syncblk entry and update the object header with the corresponding number. As the C# lock keyword expands to a try-finally that makes use of the Monitor class, a Monitor object is created on the syncblk for synchronization. A call to the GetHashCode method populates the syncblk with the object hash code.
There are other fields in the SyncBlock that are used in COM interop and for marshaling delegates to unmanaged code, but which are not relevant for a typical object usage.
TypeHandle follows the syncblk number in the ObjectInstance. In order to maintain continuity, I will discuss TypeHandle after elaborating on the instances variables. A variable list of instance fields follows the TypeHandle. By default, the instance fields will be packed in such a way that memory is used efficiently and padding is minimized for alignment. The code in Figure 7 shows a SimpleClass that has a bunch of instance variables with varying sizes contained in it.
Figure 7 SimpleClass with Instance Variables
Figure 8 shows an example of a SimpleClass object instance in the Visual Studio debugger memory window. We set a breakpoint on the return statement in Figure 7 and used the address of the simpleObj contained in the ECX register to display object instance in the memory window. The first 4-byte block is the syncblk number. As we didn’t use the instance in any synchronizing code (or access its HashCode), this is set to 0. The object reference, as stored in the stack variable, points to 4 bytes starting at offset 4. The Byte variables b1, b2, b3, and b4 are all packed side by side. Both of the short variables, s1 and s2, are packed together. The String variable str is a 4-byte OBJECTREF that points to the actual instance of the string located on the GC Heap. String is a special type in that all instances containing the same literal will be made to point to the same instance in a global string table during the assembly loading process. This process is called string interning and is designed to optimize memory usage. As we mentioned previously, in the .NET Framework 1.1, an assembly cannot opt out of this interning process, although future versions of CLR may provide this capability.

Figure 8 Debugger Memory Window for Object Instance
So the lexical sequence of member variables in the source code is not maintained in memory by default. In interop scenarios where lexical sequence has to be carried forward into memory, the StructLayoutAttribute can be used, which takes a LayoutKind enumeration as the argument. LayoutKind.Sequential will maintain the lexical sequence for the marshaled data, though in the .NET Framework 1.1 it will not affect the managed layout (however, in the .NET Framework 2.0, it will). In interop scenarios where you really need to have extra padding and explicit control of the field sequence, LayoutKind.Explicit can be combined with FieldOffset decoration at the field level.
Having looked at the raw memory contents, let’s use SOS to look at the object instance. One useful command is DumpHeap, which allows listing of all the heap contents and all the instances of a particular type. Instead of relying on the registers, DumpHeap can show the address of the only instance we created:

The total size of the object is 36 bytes. No matter how large the string is, instances of SimpleClass contain only DWORD OBJECTREF. SimpleClass’s instance variables only occupy 28 bytes. The remaining 8 bytes are comprised of the TypeHandle (4 bytes) and the syncblk number (4 bytes). Having found the address of the instance simpleObj, let’s dump the contents of this instance using the DumpObj command, as shown here:

As noted, the default layout generated for classes by the C# compiler is LayoutType.Auto (for structs, LayoutType.Sequential is used); hence the class loader rearranged the instance fields to minimize the padding. We can use ObjSize to dump the graph that includes the space taken up by the instance, str. Here’s the output:

Son of Strike
The SOS debugger extension is used to display the contents of CLR data structures in this article. It’s part of the .NET Framework installation and is located at %windir%Microsoft.NETFrameworkv1.1.4322. Before you load SOS into the process, enable managed debugging from the project properties in Visual Studio .NET. Add the directory in which SOS.dll is located to the PATH environment variable. To load SOS.dll, while at a breakpoint, open Debug | Windows | Immediate. In the immediate window, execute .load sos.dll. Use !help to get a list of debugger commands. For more information on SOS, see the June 2004 Bugslayer column.
If you subtract the size of the SimpleClass instance (36 bytes) from the overall size of the object graph (72 bytes), you should get the size of the str—that is, 36 bytes. Let’s verify this by dumping the str instance. Here’s the output:

If you add the size of the string instance str (36 bytes) to the size of SimpleClass instance (36 bytes), you get a total size of 72 bytes, as reported by the ObjSize command.

Note that ObjSize will not include the memory taken up by the syncblk infrastructure. Also, in the .NET Framework 1.1, the CLR is not aware of the memory taken up by any unmanaged resources like GDI objects, COM objects, file handles, and so on; hence, they will not be reported by this command.
TypeHandle, a pointer to the MethodTable, is located right after the syncblk number. Before an object instance is created, the CLR looks up the loaded types, loads the type if not found, obtains the MethodTable address, creates the object instance, and populates the object instance with the TypeHandle value. The JIT compiler-generated code uses TypeHandle to locate the MethodTable for method dispatching. The CLR uses TypeHandle whenever it has to backtrack to the loaded type through MethodTable.
MethodTable
Each class and interface, when loaded into an AppDomain, will be represented in memory by a MethodTable data structure. This is a result of the class-loading activity before the first instance of the object is ever created. While ObjectInstance represents the state, MethodTable represents the behavior. MethodTable binds the object instance to the language compiler-generated memory-mapped metadata structures through EEClass. The information in the MethodTable and the data structures hanging off it can be accessed from managed code through System.Type. A pointer to the MethodTable can be acquired even in managed code through the Type.RuntimeTypeHandle property. TypeHandle, which is contained in the ObjectInstance, points to an offset from the beginning of the MethodTable. This offset is 12 bytes by default and contains GC information which we will not discuss here.
Figure 9 shows the typical layout of the MethodTable. We’ll show some of the important fields of the TypeHandle, but for a more complete list, look at the figure. Let’s start with the Base Instance Size as it has direct correlation to the runtime memory profile.

Figure 9 MethodTable Layout
Base Instance Size
The Base Instance Size is the size of the object as computed by the class loader, based on the field declarations in the code. As discussed previously, the current GC implementation needs an object instance of at least 12 bytes. If a class does not have any instance fields defined, it will carry an overhead of 4 bytes. The rest of the 8 bytes will be taken up by the Object Header (which may contain a syncblk number) and TypeHandle. Again, the size of the object can be influenced by a StructLayoutAttribute.
Look at the memory snapshot (Visual Studio .NET 2003 memory window) of a MethodTable for MyClass from Figure 3 (MyClass with two interfaces) and compare it with SOS-generated output. In Figure 9, the object size is located at a 4-byte offset and the value is 12 (0x0000000C) bytes. The following is the output of DumpHeap from SOS:

Method Slot Table
Embedded within the MethodTable is a table of slots that point to the respective method descriptors (MethodDesc), enabling the behavior of the type. The Method Slot Table is created based on the linearized list of implementation methods laid out in the following order: Inherited virtuals, Introduced virtuals, Instance Methods, and Static Methods.
The ClassLoader walks through the metadata of the current class, parent class, and interfaces, and creates the method table. In the layout process, it replaces any overridden virtual methods, replaces any parent class methods being hidden, creates new slots, and duplicates slots as necessary. The duplication of slots is necessary to create an illusion that each interface has its own mini vtable. However, the duplicated slots point to the same physical implementation. MyClass has three instance methods, a class constructor (.cctor), and an object constructor (.ctor). The object constructor is automatically generated by the C# compiler for all objects having no constructors explicitly defined. Class constructor is generated by the compiler as we have a static variable defined and initialized. Figure 10 shows the layout of the method table for MyClass. The layout shows 10 methods because of the duplication of Method2 slot for IVMap, which will be covered next. Figure 11 shows the edited SOS dump of MyClass’s method table.

Figure 10 MyClass MethodTable Layout
Figure 11 SOS Dump of MyClass Method Table
The first four methods of any type will always be ToString, Equals, GetHashCode, and Finalize. These are virtual methods inherited from System.Object. The Method2 slot is duplicated, but both point to the same method descriptor. The explicitly coded .cctor and .ctor will be grouped with static methods and instance methods, respectively.
MethodDesc
Method Descriptor (MethodDesc) is an encapsulation of method implementation as the CLR knows it. There are several types of Method Descriptors that facilitate the calls to a variety of interop implementations, in addition to managed implementations. In this article we will only look at the managed MethodDesc in the context of the code shown in Figure 3. A MethodDesc is generated as a part of the class loading process and initially points to Intermediate Language (IL). Each MethodDesc is padded with a PreJitStub, which is responsible for triggering JIT compilation. Figure 12 shows a typical layout. The method table slot entry actually points to the stub instead of the actual MethodDesc data structure. This is at a negative offset of 5 bytes from the actual MethodDesc and is part of the 8-byte padding every method inherits. The 5 bytes contain instructions for a call to the PreJitStub routine. This 5-byte offset can be seen from the DumpMT output (of MyClass in Figure 11) of SOS, as MethodDesc is always 5 bytes after the location pointed to by the Method Slot Table entry. Upon the first invocation, a call to the JIT compilation routine is made. After the compilation is complete, the 5 bytes containing the call instruction will be overwritten with an unconditional jump to the JIT-compiled x86 code.

Figure 12 Method Descriptor
Disassembly of the code pointed to by the Method Table Slot entry in Figure 12will show the call to the PreJitStub. Here’s an abridged display of the disassembly before JIT for Method 2:

Now let’s execute the method and disassemble the same address:

Only the first 5 bytes at the address is code; the rest contains data of Method2’s MethodDesc. The “!u” command is unaware of this and generates gibberish, so you can ignore anything after the first 5 bytes.

CodeOrIL before JIT compilation contains the Relative Virtual Address (RVA) of the method implementation in IL. This field is flagged to indicate that it is IL. The CLR updates this field with the address of the JITed code after on-demand compilation. Let’s pick a method from the ones listed and dump the MethodDesc using DumpMT command before and after JIT compilation:

After compilation, MethodDesc looks like this:

The Flags field in the method descriptor is encoded to contain the information about the type of the method, such as static, instance, interface method, or COM implementation.

Let’s see another complicated aspect of MethodTable: Interface implementation. It’s made to look simple to the managed environment by absorbing all the complexity into the layout process. Next, we’ll show how the interfaces are laid out and how interface-based method dispatching really works.
Interface Vtable Map and Interface Map
At offset 12 in the MethodTable is an important pointer, the IVMap. As shown in Figure 9, IVMap points to an AppDomain-level mapping table that is indexed by a process-level interface ID. The interface ID is generated when the interface type is first loaded. Each interface implementation will have an entry in IVMap. If MyInterface1 is implemented by two classes, there will be two entries in the IVMap table. The entry will point back to the beginning of the sub-table embedded within the MyClass method table, as shown in Figure 9. This is the reference with which the interface-based method dispatching occurs. IVMap is created based on the Interface Map information embedded within the method table. Interface Map is created based on the metadata of the class during the MethodTable layout process. Once typeloading is complete, only IVMap is used in method dispatching.
The Interface Map at offset 28 will point to the InterfaceInfo entries embedded within the MethodTable. In this case, there are two entries for each of the two interfaces implemented by MyClass. The first 4 bytes of the first InterfaceInfo entry points to the TypeHandle of MyInterface1 (see Figure 9 and Figure 10). The next WORD (2 bytes) is taken up by Flags (where 0 is inherited from parent, and 1 is implemented in the current class). The WORD right after Flags is Start Slot, which is used by the class loader to lay out the interface implementation sub-table. For MyInterface1, the value is 4, which means that slots 5 and 6 point to the implementation. For MyInterface2, the value is 6, so slots 7 and 8 point to the implementation. ClassLoader duplicates the slots if necessary to create the illusion that each interface gets its own implementation while physically mapping to the same method descriptor. In MyClass, MyInterface1.Method2 and MyInterface2.Method2 will point to the same implementation.
Interface-based method dispatching occurs through IVMap, while direct method dispatch occurs through the MethodDesc address stored at the respective slot. As mentioned earlier, the .NET Framework uses the fastcall calling convention. The first two arguments are typically passed through ECX and EDX registers, if possible. This first argument of the instance method is always a this pointer, which is passed through the ECX register as shown by the “mov ecx, esi” statement:

These disassemblies show that the direct call to MyClass’s instance method does not use offset. The JIT compiler writes the address of the MethodDesc directly into the code. Interface-based dispatch happens through IVMap and requires a few extra instructions than the direct dispatch. One is used to fetch the address of the IVMap, and the other to fetch the start slot of the interface implementation within the Method SlotTable. Also, casting an object instance to an interface merely copies the this pointer to the target variable. In Figure 2, the statement “mi1 = mc;” uses a single instruction to copy the OBJECTREF in mc to mi1.
Virtual Dispatch
Let’s look now at Virtual Dispatch and compare it with direct and interface-based dispatch. Here is the disassembly for a virtual method call to MyClass.Method3 from Figure 3:

Virtual dispatch always occurs through a fixed slot number, irrespective of the MethodTable pointer in a given implementation class (type) hierarchy. During the MethodTable layout, ClassLoader replaces the parent implementation with the overriding child implementation. As a result, method calls coded against the parent object get dispatched to the child object’s implementation. The disassembly shows that the dispatch occurs through slot number 8 in the debugger memory window (as seen in Figure 10) as well as the DumpMT output.

Static Variables
Static variables are an important constituent part of the MethodTable data structure. They are allocated as a part of the MethodTable right after the method table slot array. All the primitive static types are inlined while the static value objects like structs and reference types are referred through OBJECTREFs created in the handle tables. OBJECTREF in the MethodTable refers to OBJECTREF in the AppDomain handle table, which refers to the heap-created object instance. Once created, OBJECTREF in the handle table will keep the object instance on the heap alive until the AppDomain is unloaded. In Figure 9, a static string variable, str, points to OBJECTREF on the handle table, which points to MyString on the GC Heap.
EEClass
EEClass comes to life before the MethodTable is created and, when combined with MethodTable, is the CLR version of a type declaration. In fact, EEClass and MethodTable are logically one data structure (together they represent a single type), and were split based on frequency of use. Fields that get used a lot are in MethodTable, while fields that get used infrequently are in EEClass. Thus information (like names, fields, and offsets) needed to JIT compile functions end up in EEClass, however info needed at run time (like vtable slots and GC information) are in MethodTable.
There will be one EEClass for each type loaded into an AppDomain. This includes interface, class, abstract class, array, and struct. Each EEClass is a node of a tree tracked by the execution engine. CLR uses this network to navigate through the EEClass structures for purposes including class loading, MethodTable layout, type verification, and type casting. The child-parent relationship between EEClasses is established based on the inheritance hierarchy, whereas parent-child relationships are established based on the combination of inheritance hierarchy and class loading sequence. New EEClass nodes get added, node relationships get patched, and new relationships get established as the execution of the managed code progresses. There is also a horizontal relationship with sibling EEClasses in the network. EEClass has three fields to manage the node relationships between loaded types: ParentClass, SiblingChain, and ChildrenChain. Refer to Figure 13 for the schematics of EEClass in the context of MyClass from Figure 4.
Figure 13 shows only a few of the fields relevant to this discussion. Because we’ve omitted some fields in the layout, we have not really shown the offsets in this figure. EEClass has a circular reference to MethodTable. EEClass also points MethodDesc chunks allocated on HighFrequencyHeap of the default AppDomain. A reference to a list of FieldDesc objects allocated on the process heap provides field layout information during MethodTable construction. EEClass is allocated on the LowFrequencyHeap of the AppDomain so that the operating system can better perform page management of memory, thereby reducing the working set.

Figure 13 EEClass Layout
Other fields shown in Figure 13 are self-explanatory in the context of MyClass (Figure 3). Let’s look now at the real physical memory by dumping the EEClass using SOS. Run the program from Figure 3after setting a breakpoint on the line, mc.Method1. First obtain the address of EEClass for MyClass using the command Name2EE:

The first argument to Name2EE is the module name that can be obtained from DumpDomain command. Now that we have the address of the EEClass, we’ll dump the EEClass itself:

Figure 13 and the DumpClass output look essentially the same. Metadata token (mdToken) represents the MyClass index in the memory mapped metadata tables of the module PE file, and the Parent class points to System.Object. Sibling Chain (Figure 13) shows that it is loaded as a result of the loading of the Program class.
MyClass has eight vtable slots (methods that can be virtually dispatched). Even though Method1 and Method2 are not virtual, they will be considered virtual methods when dispatched through interfaces and added to the list. Add .cctor and .ctor to the list, and you get 10 (0?a) total methods. The class has two static fields that are listed at the end. MyClass has no instance fields. The rest of the fields are self-explanatory.
Conclusion
That concludes our tour of the some of the most important internals of the CLR. Obviously, there’s much more to be covered, and in much more depth, but we hope this has given you a glimpse into how things work. Much of the information presented here will likely change with subsequent releases of the CLR and the .NET Framework. But although the CLR data structures covered in this article may change, the concepts should remain the same.
Hanu Kommalapati is an Architect at Microsoft Gulf Coast District (Houston). In his current role at Microsoft, he helps enterprise customers in building scalable component frameworks based on the .NET Framework. He can be reached at hanuk@microsoft.com.
Tom Christian is an Escalation Engineer with Developer Support at Microsoft, working with ASP.NET and the .NET debugger extension for WinDBG (sos/psscor). He is based in Charlotte, NC and can be contacted at tomchris@microsoft.com.