Note: this article uses the old project.json format of .net core, i havent gotten around to updating this yet.

In this article I am going to go through step by step on how to get started with Visual Studio Code and creating your first C# based AWS Lambda function out of it. As Visual Studio Code is cross-platform you can be on MAC, Windows or Linux, in this article I am going to be on Windows, however that should effect only the shortcuts that I mention to open different windows, you should still be able to follow through.

What is Visual Studio Code? How is that different from Visual Studio?

Visual studio code is a free and open source code editor developed by Microsoft. It is cross-platform and works with Windows, MAC and Linux, for more features and background on this checkout this wiki link.

In my experience (limited to a few months) with this code editor, it is by far the best free code editor that I have worked with. Its ability to go multi-platform along with being free to use while supporting all the languages that I will ever use (with plugins installed) it is a go-to IDE for anyone on a budget or a requirement of cross-platform targeting.

What is AWS Lambda?

AWS Lambda is a compute service by the AWS cloud offering, it is fully managed and with it there is no need to provision or manage servers (think patching, updates etc). In AWS Lambda basically you upload a function which can receive data through either events invoked by other AWS services or custom ones that you create or can even be invoked via a HTTP call with AWS API gateway sitting at front. The latter configuration basically means that you can deploy an entire application server-less, with you paying only by the time that your function needs to run coupled with the resource that you configure that each function receives, check out this AWS link for its detailed pricing.

To visualize what this means, think of Lambda function as a single http endpoint e.g. [GET] http://myApi.com/pets/ would be one Lambda function, while [PUT] http://myApi.com/pets/ will be another.

Interesting… where can I use this?

You can use AWS Lambda for building your microservices architecture by going completely serverless and avoiding a lot of complications that come from the microservices architecture.

You can also use this in conjunction with other AWS services like AWS Kinesis (data streaming service) to process data before it reaches its destination(s).

You can also use AWS Lambda as a scheduled job by driving it through scheduled Cloudwatch events.

Step 1: Install .Net core and VS code extensions

If you don’t have .Net core installed get it from here.

Install the c# extension for VS code from VS code marketplace.

This tutorial will be most useful if you have an AWS account ready with near-Admin priviliges and Visual Studio Code opened in front of you along with ~3 hours of your time for exploration as you follow the steps.

Step 2: Initialize the project

1. Create a folder, I called it LogProcessor.Lambda (Yes, I am bad with names).
2. Open the created folder in VS code from the File Menu > Open Folder
3. Press ctrl+~ (the key besides the 1 number), that should open a command prompt and write dotnet new This command will populate your local package cache if its run for the first time, create two files in the system program.cs (entry point to the program) and project.json (contains dependencies for the project), you should see something similar to the screenshot below:

Step 3: Preparing a shell for Lambda to execute

The AWS Lambda runtime does not use the Main() method in the program.cs file created by the initialize command in the previous step.

Open the program.cs file and replace the contents with this:

using System;
using System.IO;

namespace CSharpLambdaFunction
{
    public class LambdaHandler
    {
        public static void Main()
        {
            //we will use this for debugging later
        }

        public Stream LogHandler(Stream inputStream)
        {
            //this is where lambda function logic will go later.
            throw new NotImplementedException();
        }
    }
}

Open the command prompt at this location again (ctrl+~) and run the command dotnet restore, this will restore all the dependencies into the project.

Try running the command dotnet publish, this should successfully compile and put the binaries at a folder which it will tell you, see screenshot below:

Step 4: Setup debugging

Click on the debug icon on the left pane as shown below:

Click on the settings icon and select .Net core as the debug environment as shown below:

Click on the play button at the top debug bar and it will ask you to configure a task runner, click on Configure Task Runner and select .Net core from the list of available runners as shown below:

Open Launch.json and change the program attribute inside it to point to your assembly, like so:

Put a break point (where you want the code to stop while debugging) at the main method in program.cs file, then press the start debug button (the play icon, topbar) from the debug pane to start debugging, you should see something like below:

Step 5: Prepare your code

For testing purpose, I have added a testData.json file into my project, see image below on how to do this:

Next paste this into the test data file:

{
    "requestId": "79ea2ddb-b52f-42ff-b55b-9a1a3ff09b0a",
    "sessionId": "4c1149bc-b947-42d3-8a9d-4d052b20862c",
    "timestamp": "2017-01-17T02:09:36.4952363Z",
    "userId": "d4406b60-8439-424c-80c5-ba096728b3cd",
    "serverIp": "192.168.1.1",
    "message": "The request had bad words in it",
    "level": "Module",
    "category": "Information",
    "request": "{\"iAm\": \"a request\"}",
    "response": "{\"andHereIs\": \"a response\"}"
}

Hint: You can press shift+alt+F to auto format your code in VS code.

Copy and paste the following code in your program.cs file:

using System;
using System.IO;
using System.Threading.Tasks;
using Amazon.S3;
using Amazon.S3.Model;

namespace CSharpLambdaFunction
{
    public class LambdaHandler
    {
        public static void Main()
        {
            Task.Run(async () =>
            {
                using (var stream = new FileStream($"{Directory.GetCurrentDirectory()}\\testData.json", FileMode.Open))
                {
                    await new LambdaHandler().LogHandler(stream);
                }
            }).GetAwaiter().GetResult();
        }

        public async Task LogHandler(Stream inputStream)
        {
            LogEntry entry;
            using (var reader = new StreamReader(inputStream))
            {
                entry = Deserialize(reader);
            }
            var reqTask = SendToS3AndGetLink(entry.Request);
            var resTask = SendToS3AndGetLink(entry.Response);

            entry.Request = await reqTask;
            entry.Response = await resTask;

            return ConvertToStream(entry);
        }

        private LogEntry Deserialize(TextReader reader)
        {
            var serializer = new Newtonsoft.Json.JsonSerializer();

            return (LogEntry)serializer.Deserialize(reader, typeof(LogEntry));
        }

        private Stream ConvertToStream(LogEntry logEntry)
        {
            var serializer = new Newtonsoft.Json.JsonSerializer();

            var memStream = new MemoryStream();
            var writer = new StreamWriter(memStream);
            serializer.Serialize(writer, logEntry);

            writer.Flush();

            return memStream;
        }

        private Stream ConvertToStream(string value)
        {
            var memStream = new MemoryStream();
            var writer = new StreamWriter(memStream);

            writer.Write(value);
            writer.Flush();

            return memStream;
        }

        private async Task<string> SendToS3AndGetLink(string value)
        {
            var s3 = new AmazonS3Client(Amazon.RegionEndpoint.USWest2); //define your own region here

            var putRequest = new PutObjectRequest();

            var key = Guid.NewGuid().ToString().Replace("-", string.Empty);
            putRequest.BucketName = GetBucketName();
            putRequest.Key = key;
            putRequest.InputStream = ConvertToStream(value);
            putRequest.ContentType = "application/json";

            var response = await s3.PutObjectAsync(putRequest);

            var urlRequest = new GetPreSignedUrlRequest();
            urlRequest.BucketName = GetBucketName();
            urlRequest.Expires = DateTime.UtcNow.AddYears(2);
            urlRequest.Key = key;

            return s3.GetPreSignedURL(urlRequest);
        }

        private string GetBucketName()
        {
            return "cloudncode-logs";
        }
    }

    public class LogEntry
    {
        public string RequestId { get; set; }
        public string SessionId { get; set; }
        public string Timestamp { get; set; }
        public string UserId { get; set; }
        public string ServerIp { get; set; }
        public string Message { get; set; }
        public string Level { get; set; }
        public string Category { get; set; }
        public string Request { get; set; }
        public string Response { get; set; }
    }
}

*Note that this is clearly not production quality code, that is not the purpose of this article, in a real working code you would want the bucket name especially to come from config

**Notice that I have defined a region endpoint as US-west2 in the code, change that in case your region differs

***I have the S3 bucket name as cloudncode-logs, take a note, in some time we will be creating this bucket, if you give a different name there, make sure to change this in the code as well

Open project.json and add dependencies used above, the final file should look similar to:

{
  "version": "1.0.0-*",
  "buildOptions": {
    "debugType": "portable",
    "emitEntryPoint": true,
    "copyToOutput":{
      "includeFiles": "testData.json"
    }
  },
  "dependencies": {
    "AWSSDK.Core": "3.3.7.1",
    "AWSSDK.S3": "3.3.5.2",
    "Newtonsoft.Json": "9.0.1"
  },
  "tools": {
    "Amazon.Lambda.Tools": "1.0.1-preview1"
  },
  "frameworks": {
    "netcoreapp1.0": {
      "dependencies": {
        "Microsoft.NETCore.App": {
          "type": "platform",
          "version": "1.0.1"
        }
      },
      "imports": "dnxcore50"
    }
  }
}

Amazon.Lambda.Tools in the tools section above will help us in uploading our lambda function later.

Install the packages by running dotnet restore in the command line (ctrl+~).

Step 6: Add AWS users and roles

Roles in AWS are a nice way to avoid putting credentials in the code itself, we will add a role which our lambda function will use to successfully fire the AWS API’s used in our code.

First, login into the AWS console, go to the security credentials section and click on roles, then create role…. see the image below:

Write in a name for the role:

Select AWS Lambda service role:

Type “s3” in the search box and select AmazonS3FullAccess, in the real world, you don’t want to give full access, create a custom policy and attach that to the role, but for our testing, this is fine.

We will use this role to run our lambda function with so as to avoid hardcoding credentials, now we need to run the same code on our local as well… for this let’s add an IAM user first for ourselves to get access key and secret. It is important to note that although you can generate the access keys for root accounts as well, you shouldn’t do that, it allows unrestricted access which cannot be restricted at any time without deleting the key itself.

Let’s go to the User tab from the left pane as in the screenshot below:

Give the user a name and check the programmatic access checkbox, you need this to get access keys, other options are for console access via the browser.

Click on attach existing policies, type s3 in the search box and select AmazonS3FullAccess and then type lambda and add AWSLambdaFullAccess as well, the first permission will allow us to debug the code and objects into S3 while the second will let us create the Lambda function right from VS code which we will do later in this guide.

Create the user, but before you close make sure you download the CSV file containing your credentials, you won’t get this option again

Now, to configure the code to take these credentials while still avoiding hardcoding, go to C:\Users.aws\credentials, make the DIR and the file if they don’t exist. In windows if you get an error while creating the DIR .aws, go to the command prompt, navigate to the C:\Users folders and then run mkdir .aws.

Note that the credentials file is in lowercase and has no extension.

Paste in the following content:

[default]
aws_access_key_id = {accessKey}
aws_secret_access_key = {secret}

The above key and secret are the keys that you downloaded in the CSV when you created the user

For more information on how this works, visit AWS documentation at this link.

Step 7: Create a S3 bucket

Go to the S3 service home in the AWS console

Type in the bucket name, make sure the GetBucketName() function in program.cs is returning the same bucket name whatever you enter here.

Note that whatever region you choose here has to be the same one in the SendToS3AndGetLink function in program.cs. For a complete list of AWS regions and endpoints go to AWS documentation on this link.

Step 8: Run your code

Open the command prompt in VS code and type in dotnet restore to dlownload all the packages mentioned in project.json file that we had created earlier.

Press F5 to run your code, if you followed every step correctly the program would exit after a few seconds or you can debug through to see what is happening. Go to the S3 Bucket in AWS console and you should find your two files successfully uploaded there.

For troubleshooting, you can open the debug console from the View menu in VS code, it should print out errors in detail over there.

Ok great, we have so far:

1. Learned basics of working with .Net core
2. Created a basic lambda function code
3. Learned to debug through it
4. Learned about AWS users & roles enough to figure out how to avoid hardcoding security credentials necessary for effective debugging.

Up next, we will actually create the Lambda function and test if thats running fine with dynamic input.

Step 8: Creating the Lambda function

If you were paying close attention to the project.json you would have noticed that we had added a tools section with Amazon.Lambda.Tools present there, this basically adds some commands into the dotnet-cli which we will use to create the Lambda function in AWS.

Open command prompt (ctrl+~) and type in dotnet lambda help and you should see the following:

Type in dotnet lambda package, this will guide you on steps to create the package which we will upload soon, note the location of the zip file in the output.

Note that you could also use the deploy-function to deploy it directly from VS code, however the Amazon.Lambda.Tools is still in preview mode and Amazon is yet to update some code there, at the time of writing this

article it was throwing an exception that it does not recognize the C# runtime (which Lambda actually does).

Open the lambda home page in the AWS console, click on create function and select the blank function in the next screen

On the next screen, click next, then on the next enter the details as the screen asks for it. Notable tihngs here are:

1. Choose the runtime as C#
2. Upload the zip file that you had packaged earlier
3. In the Handler text box, enter this LogProcessor.Lambda::CSharpLambdaFunction.LambdaHandler::LogHandler, its basically ::::
4. Keep the memory at 128MB, you wouldn’t need more.
5. Set the timeout to 30 secs

For the rest refer the screenshots below:

On the next screen, review and click on Create Function.

Step 9: Test your Lambda function

Go to the Lambda home screen and click on the function name that you just created.

Click on Actions and click on Configure Test Event

Paste in the contents of testData.json here, refer screenshot:

Click on the Test button.

There you go, your Lambda function is done! You can now use this Lambda function to transform logs coming from Kinesis or create other functions which can act as Microservices via the AWS API Gateway with AWS Lambda as backend and lots of other things in your serverless journey.

Download the source code used in this tutorial here.

What’s next?

This article will be used in building our log data pipeline (upcoming article) part of the series where I talk about “What should you log in an application and how to avoid having 24×7 support looking at them“.

Follow this blog/subscribe to email alerts (top of sidebar or below comments in mobile) so that you know when future posts in this series come about, I will be talking about using AWS Kinesis Firehose as the log data pipeline to put data from your servers into AWS Elastic Search service next.