Dynamically compile a class in App_Code while pre-compiling the rest of the project/library

Problem Overview

We need to overcome two different problems here:

1. The first is having a single file that can be compiled at build time and also re-compiled at runtime.
2. The second is resolving the two different versions of that class created by the solving the first problem so we can actually make use of them.

Problem 1 - Schrödinger's Compilation

The first problem is trying to get a class that is both compiled and not compiled. We need to compile it at design time so that other sections of code are aware it exists and can use its properties with strong typing. But normally, compiled code is stripped out of the output so there aren't multiple versions of the same class causing naming conflicts.

In any case, we need to compile the class initially, but there are two options to persist a re-compilable copy:

1. Add the file to App_Code, which is compiled at runtime by default, but set it's Build Action = Compile so it's available at design time as well.
2. Add a regular class file, which is compiled at design time by default, but set it to Copy to Output Directory = Copy Always, so there's a chance we can evaluate it at runtime as well.

Problem 2 - Self Imposed DLL Hell

At a bare minimum, this is a tricky task to charge to the compiler. Any code that consumes a class, must have a guarantee that it exists at compile time. Anything that is dynamically compiled, whether via App_Code or otherwise, will be part of an entirely different assembly. So producing an identical class is treated more like a picture of that class. The underlying type might be the same, but ce n'est une pipe.

We have two options: use an interface or crosswalk between assemblies:

1. If we use an interface, we can compile it with the initial build and any dynamic types can implement that same interface. This way we are safely relying on something that exists at compile time, and our created class can be safely swapped out as a backing property.

2. If we cast types across assemblies, it's important to note that any existing usages rely on the type that was originally compiled. So we'll need to grab the values from the dynamic type and apply those property values to the original type.

Existing Answers

Per evk, I like the idea of querying AppDomain.CurrentDomain.GetAssemblies() on startup to check for any new assemblies/classes. I'll admit that using an interface is probably an advisable way to unify precompiled/dynamically compiled classes, but I would ideally like to have a single file/class that can simply be re-read if it changes.

Per S.Deepika, I like the idea of dynamically compiling from a file, but don't want to have to move the values to a separate project.

Ruling out App_Code

App_Code does unlock the ability to build two versions of the same class, but it's actually hard to modify either one after publication as we'll see. Any .cs file located in ~/App_Code/ will be dynamically compiled when the application runs. So in Visual Studio, we can build the same class twice by adding it to App_Code and setting the Build Action to Compile.

Build Action and Copy Output:

When we debug locally, all .cs files will be built into the project assembly and the physical file in ~/App_Code will also be built as well.

We can identify both types like this:

// have to return as object (not T), because we have two different classespublic List<(Assembly asm, object instance, bool isDynamic)> FindLoadedTypes<T>(){    var matches = from asm in AppDomain.CurrentDomain.GetAssemblies()                  from type in asm.GetTypes()                  where type.FullName == typeof(T).FullName                  select (asm,                      instance: Activator.CreateInstance(type),                      isDynamic: asm.GetCustomAttribute<GeneratedCodeAttribute>() != null);    return matches.ToList();}var loadedTypes = FindLoadedTypes<Apple>();

Compiled and Dynamic Types:

This is really close to solving problem #1. We have access to both types every time the app runs. We can use the compiled version at design time and any changes to the file itself will automatically be recompiled by IIS into a version that we can access at runtime.

The problem is apparent however once we step out of debug mode and try to publish the project. This solution relies on IIS building the App_Code.xxxx assembly dynamically, and that relies on the .cs file being inside the root App_Code folder. However, when a .cs file is compiled, it is automatically stripped out of the published project, to avoid the exact scenario we're trying to create (and delicately manage). If the file was left in, it would produce two identical classes, which would create naming conflicts whenever either one was used.

We can try to force its hand by both compiling the file into the project's assembly and also copying the file into the output directory. But App_Code doesn't work any of it's magic inside of ~/bin/App_Code/. It'll only work at the root level ~/App_Code/

App_Code Compilation Source:

With every publish, we could manually cut and paste the generated App_Code folder from the bin and place it back at the root level, but that's precarious at best. Perhaps we could automate that into build events, but we'll try something else...

Solution

Compile + (Copy to Output and Manually Compile File)

Let's avoid the App_Code folder because it will add some unintended consequences.

Just create a new folder named Config and add a class that will store the values we want to be able to modify dynamically:

~/Config/AppleValues.cs:

public class Apple{    public string StemColor { get; set; } = "Brown";    public string LeafColor { get; set; } = "Green";    public string BodyColor { get; set; } = "Red";}

Again, we'll want to go to the file properties (F4) and set to compile AND copy to output. This will give us a second version of the file we can use later.

We'll consume this class by using it within a static class that exposes the values from anywhere. This helps separate concerns, especially between the need to dynamically compile and statically access.

~/Config/GlobalConfig.cs:

public static class Global{    // static constructor    static Global()    {        // sub out static property value        // TODO magic happens here - read in file, compile, and assign new values        Apple = new Apple();    }    public static Apple Apple { get; set; }}

And we can use it like this:

var x = Global.Apple.BodyColor;

What we'll attempt to do inside the static constructor, is seed Apple with the values from our dynamic class. This method will be called once every time the application is restarted, and any changes to the bin folder will automatically trigger recycling the app pool.

In short order, here's what we'll want to accomplish inside of the constructor:

string fileName = HostingEnvironment.MapPath("~/bin/Config/AppleValues.cs");var dynamicAsm = Utilities.BuildFileIntoAssembly(fileName);var dynamicApple = Utilities.GetTypeFromAssembly(dynamicAsm, typeof(Apple).FullName);var precompApple = new Apple();var updatedApple = Utilities.CopyProperties(dynamicApple, precompApple);// set static propertyApple = updatedApple;

fileName - The File path might be specific to where you'd like to deploy this, but note that inside of a static method, you need to use HostingEnvironment.MapPath instead of Server.MapPath

BuildFileIntoAssembly - In terms of loading the assembly from a file, I've adapted the code from the docs on CSharpCodeProvider and this question on How to load a class from a .cs file. Also, rather than fight dependencies, I just gave the compiler access to every assembly that was currently in the App Domain, same as it would have gotten on the original compilation. There's probably a way to do that with less overhead, but it's a one time cost so who cares.

CopyProperties - To map the new properties onto the old object, I've adapted the method in this question on how to Apply properties values from one object to another of the same type automatically? which will use reflection to break down both objects and iterate over each property.

Utilities.cs

Here's the full source code for the Utility methods from above

public static class Utilities{    /// <summary>    /// Build File Into Assembly    /// </summary>    /// <param name="sourceName"></param>    /// <returns>https://msdn.microsoft.com/en-us/library/microsoft.csharp.csharpcodeprovider.aspx</returns>    public static Assembly BuildFileIntoAssembly(String fileName)    {        if (!File.Exists(fileName))            throw new FileNotFoundException($"File '{fileName}' does not exist");        // Select the code provider based on the input file extension        FileInfo sourceFile = new FileInfo(fileName);        string providerName = sourceFile.Extension.ToUpper() == ".CS" ? "CSharp" :                              sourceFile.Extension.ToUpper() == ".VB" ? "VisualBasic" : "";        if (providerName == "")            throw new ArgumentException("Source file must have a .cs or .vb extension");        CodeDomProvider provider = CodeDomProvider.CreateProvider(providerName);        CompilerParameters cp = new CompilerParameters();        // just add every currently loaded assembly:        // https://stackoverflow.com/a/1020547/1366033        var assemblies = from asm in AppDomain.CurrentDomain.GetAssemblies()                         where !asm.IsDynamic                         select asm.Location;        cp.ReferencedAssemblies.AddRange(assemblies.ToArray());        cp.GenerateExecutable = false; // Generate a class library        cp.GenerateInMemory = true; // Don't Save the assembly as a physical file.        cp.TreatWarningsAsErrors = false; // Set whether to treat all warnings as errors.        // Invoke compilation of the source file.        CompilerResults cr = provider.CompileAssemblyFromFile(cp, fileName);        if (cr.Errors.Count > 0)            throw new Exception("Errors compiling {0}. " +                string.Join(";", cr.Errors.Cast<CompilerError>().Select(x => x.ToString())));        return cr.CompiledAssembly;    }    // have to use FullName not full equality because different classes that look the same    public static object GetTypeFromAssembly(Assembly asm, String typeName)    {        var inst = from type in asm.GetTypes()                   where type.FullName == typeName                   select Activator.CreateInstance(type);        return inst.First();    }    /// <summary>    /// Extension for 'Object' that copies the properties to a destination object.    /// </summary>    /// <param name="source">The source</param>    /// <param name="target">The target</param>    /// <remarks>    /// https://stackoverflow.com/q/930433/1366033    /// </remarks>    public static T2 CopyProperties<T1, T2>(T1 source, T2 target)    {        // If any this null throw an exception        if (source == null || target == null)            throw new ArgumentNullException("Source or/and Destination Objects are null");        // Getting the Types of the objects        Type typeTar = target.GetType();        Type typeSrc = source.GetType();        // Collect all the valid properties to map        var results = from srcProp in typeSrc.GetProperties()                      let targetProperty = typeTar.GetProperty(srcProp.Name)                      where srcProp.CanRead                         && targetProperty != null                         && (targetProperty.GetSetMethod(true) != null && !targetProperty.GetSetMethod(true).IsPrivate)                         && (targetProperty.GetSetMethod().Attributes & MethodAttributes.Static) == 0                         && targetProperty.PropertyType.IsAssignableFrom(srcProp.PropertyType)                      select (sourceProperty: srcProp, targetProperty: targetProperty);        //map the properties        foreach (var props in results)        {            props.targetProperty.SetValue(target, props.sourceProperty.GetValue(source, null), null);        }        return target;    }}

But Why Tho?

Okay, so there are other more conventional ways to accomplish the same goal. Ideally, we'd shoot for Convention > Configuration. But this provides the absolute easiest, most flexible, strongly typed way to store config values I've ever seen.

Normally config values are read in via an XML in an equally odd process that relies on magic strings and weak typing. We have to call MapPath to get to the store of value and then do Object Relational Mapping from XML to C#. Instead here, we have the final type from the get go, and we can automate all of the ORM work between identical classes that just happen to be compiled against different assemblies.

In either case, the dream output of that process is to be able to write and consume C# directly. In this case, if I want to add an extra, fully configurable property, it's as easy as adding a property to the class. Done!

It will be available immediately and recompiled automatically if that value changes without needing to publish a new build of the app.

Dynamically Changing Class Demo:

Here's the full, working source code for the project:

Compiled Config - Github Source Code | Download Link

You can move configuration part to separate project, and create common interface like (IApplicationConfiguration.ReadConfiguration) to access it.

You can compile the code dynamically at run time like below, and you can access the configuration details using reflection.

public static Assembly CompileAssembly(string[] sourceFiles, string outputAssemblyPath){    var codeProvider = new CSharpCodeProvider();    var compilerParameters = new CompilerParameters    {        GenerateExecutable = false,        GenerateInMemory = false,        IncludeDebugInformation = true,        OutputAssembly = outputAssemblyPath    };    // Add CSharpSimpleScripting.exe as a reference to Scripts.dll to expose interfaces    compilerParameters.ReferencedAssemblies.Add(Assembly.GetExecutingAssembly().Location);    var result = codeProvider.CompileAssemblyFromFile(compilerParameters, sourceFiles); // Compile    return result.CompiledAssembly;}

Let's see how dynamic compilation of files in App_Code works. When first request to your application arrives, asp.net will compile code files in that folder into assembly (if were not compiled before), and then load that assembly into current application domain of asp.net application. That's why you see your message in a watch - assembly was compiled and is available in current app domain. Because it was compiled dynamically, of course you have compile-time error when trying to reference it explicitly - this code is not yet compiled, and when it will be compiled - it might have completely different structure and message you reference might just not be there at all. So there is no way you can explicitly reference code from dynamic-generated assembly.

What options do you have then? For example, you can have an interface for your messages:

// this interface is located in your main application code,// not in App_Code folderpublic interface IMessages {    string CodeNotFound { get; }}

Then, in your App_Code file - implement that interface:

// this is in App_Code folder, // you can reference code from main application here, // such as IMessages interfacepublic class Messages : IMessages {    public string CodeNotFound    {        get { return "The entered code was not found"; }    }}

And then in main application - provide a proxy by searching current app domain for assembly with type that implements IMessage interface (only once, then cache it) and proxy all calls to that type:

public static class Messages {    // Lazy - search of app domain will be performed only on first call    private static readonly Lazy<IMessages> _messages = new Lazy<IMessages>(FindMessagesType, true);    private static IMessages FindMessagesType() {        // search all types in current app domain        foreach (var asm in AppDomain.CurrentDomain.GetAssemblies()) {            foreach (var type in asm.GetTypes()) {                if (type.GetInterfaces().Any(c => c == typeof(IMessages))) {                    return (IMessages) Activator.CreateInstance(type);                }            }        }        throw new Exception("No implementations of IMessages interface were found");    }    // proxy to found instance    public static string CodeNotFound => _messages.Value.CodeNotFound;}

This will achieve your goal - now when you change code in App_Code Messages class, on next request asp.net will tear down current application domain (first waiting for all pending requests to finish), then create new app domain, recompile your Messages and load into that new app domain (note that this recreating of app domain always happen when you change something in App_Code, not just in this particular situation). So next request will already see new value of your message without you explicitly recompile anything.

Note that you obviously cannot add or remove messages (or change their names) without recompiling main application, because doing that will require changes to IMessages interface which belongs to main application code. If you try - asp.net will throw compilation failure error on next (and all subsequent) requests.

I'd personally avoid doing such things, but if you are fine with that - why not.