Next, if you look in the “API Gateway VTL template to put events into above DynamoDB table” section (line 193), you’ll

A full serverless.yml config file for this can be found in this gist. Ultimately, the bulk of this is CloudFormation within Serverless Framework config (if there’s a plugin I missed, or some more direct Serverless way to do it, let me know!). I refer to line numbers from this gist below. The key parts are:

First up is Cognito (line 51). If you do not need authentication on your API, you can skip this. There is a fair bit of setup, at the beginning of the resources section to configure a user pool and the policies needed for this.

Impost syndrome can have you stagnant, hopeless and questioning yourself. However, part of transitioning into Data Science is about recognizing the qualities and experiences that make you qualified for the role you desire. For me, being the captain of every single football team I’ve played for is what gives me the confidence to be vocal, and as a postman I developed the ability to be extremely thorough and pay close attention to detail which helps me when I am trying to code an algorithm (such as in the Algorithms From Scratch Series) or when I am carrying out error analysis and more.

By having TypeScript, a linter, and a formatter, you will have a better experience while developing code. But those tools are not completely automated. Sometimes you will need to run a command in the console to run them. We sometimes forget to do it and end up pushing some bad code to our repositories, and this is where this tool shines.

The key here is to package your difference, there is no point stating on your CV “I have a lot of grit” or “I am willing to learn” because these are generic things that I personally think should be expected of someone in a technical field.

A quick update since I originally published this story. Ben Duong pointed out a Serverless Framework plugin Serverless Apigateway Service Proxy. However, it doesn’t support DynamoDB’s batch updates, so cannot be used in this case. I’m also not sure on how it handles auth needs. However, if you are simply taking a single event/record into your API, it should cover it.

Think about it, if you saw someone that built a car from scratch and they have not followed a linear path into car manufacturing, but the car is as good, or if not better, than someone who did, wouldn’t you be intrigued by how they did it? Wouldn’t you be happy to have this person on your team, knowing that with limited resources and untraditional education/experience, they are resilient enough to seek solutions? Exactly.

http://molos.bodasturias.com/qxz/video-sibir-novorossiisk-v-lokomotiv-iaroslavl-v-ru-ru-1mgo-26.php

http://elta.actiup.com/cqn/video-Huracan-Argentinos-Juniors-v-en-gb-1dju30122020-.php

http://old.cocir.org/media/qas/videos-brynas-v-oskarshamn-v-sw-sw-1wyn-13.php

http://molos.bodasturias.com/qxz/video-sibir-novorossiisk-v-lokomotiv-iaroslavl-v-ru-ru-1oet-5.php

http://molos.bodasturias.com/qxz/v-ideos-sibir-novorossiisk-v-lokomotiv-iaroslavl-v-ru-ru-1yap-6.php

http://elta.actiup.com/cqn/video-Huracan-Argentinos-Juniors-v-en-gb-1prx30122020-21.php

http://old.cocir.org/media/qas/video-Rodovre-Mighty-Bulls-Aalborg-Pirates-v-en-gb-1tzf-.php

http://team.vidrio.org/xpy/video-HC-Verva-Litvinov-HC-Energie-Karlovy-Vary-v-en-gb-qvf30122020-.php

http://elta.actiup.com/cqn/videos-Huracan-Argentinos-Juniors-v-en-gb-1tgv30122020-2.php

http://old.cocir.org/media/qas/videos-Rodovre-Mighty-Bulls-Aalborg-Pirates-v-en-gb-1nci-11.php

http://main.ruicasa.com/tgq/video-AEK-Athens-Hapoel-Unet-Holon-v-en-gb-oqw-.php

http://team.vidrio.org/xpy/video-HC-Verva-Litvinov-HC-Energie-Karlovy-Vary-v-en-gb-bte-.php

http://main.ruicasa.com/tgq/video-AEK-Athens-Hapoel-Unet-Holon-v-en-gb-xkz-.php

http://team.vidrio.org/xpy/v-ideos-HC-Verva-Litvinov-HC-Energie-Karlovy-Vary-v-en-gb-kma-.php

http://old.cocir.org/media/qas/videos-Rodovre-Mighty-Bulls-Aalborg-Pirates-v-en-gb-1tfd30122020-4.php

http://main.ruicasa.com/tgq/v-ideos-AEK-Athens-Hapoel-Unet-Holon-v-en-gb-xoe-.php

http://elta.actiup.com/cqn/Video-huracan-v-argentinos-juniors-v-es-ar-1klt-13.php

http://team.vidrio.org/xpy/Video-Vaasan-Sport-Tappara-Tampere-v-en-gb-ejy-.php

http://elta.actiup.com/cqn/videos-huracan-v-argentinos-juniors-v-es-ar-1nmo-8.php

http://team.vidrio.org/xpy/video-Vaasan-Sport-Tappara-Tampere-v-en-gb-sra30122020-.php

’t designed for the students to become geniuses or billionaires, both of which apply the same formula of monomania. After all, the word “genius” is a verb, not a noun. It would therefore be impractical to expect an institution of higher learning to teach puppy-eyed pupils to embrace a life of monomania.

This alleviates the need for a Lambda, and avoids the cost of that. Not that Lambda is that expensive, but if this does wind up scaling to say millions (or hundreds of millions) of events per day, then that will be a meaningful savings. Furthermore, this is more maintainable and a simpler architecture, as it’s one less component to build and maintain.

The use case I have is an authenticated web API that takes in a potentially significant volume of events from mobile devices. This data will be stored in DynamoDB. As an additional constraint, the mobile app is sending via regular HTTP web calls, and doesn’t have the ability to use GraphQL (i.e. AppSync isn’t a possibility for this case). Finally, I want this particular API to be simple and very fast, and all the (time consuming) processing of the data will be done async. Thus, we can simply have the data come in via API Gateway and get injected directly into DynamoDB (with some basic data transformation, and integration of the user’s ID).

To me, the interesting parts for this whole thing really come down to how to do the VTL mapping template (i.e. take an incoming HTTP request’s payload and transform it to what DynamoDB needs to do an insert, and how to get the Cognito user ID and include that in the data (since all the authentication is happening “automatically” for you via API Gateway’s Cognito integration). Well, and of course how to do this all in code/Serverless instead of via the AWS console.

A key thing to note is that we use batch writes for Dynamo. These are limited to 25 items at a time. As such, our mobile clients are limited to sending events in batches of 25. But, the key is that it’s batch, even if it’s a batch with just one event. You’ll see more on this below with the VTL template iterating the incoming events.

In order to use your difference to your advantage, you have to be able to package your experience in a way that sits well with the hiring manager and company. You may not have the academics or experience to back you, but something undeniable you have is grit, but you must package this grit into something tangible.

AWS’ API Gateway allows you to directly connect it to/proxy many other AWS services. This article discusses doing this with DynamoDB, as a way to create an API that adds data to DynamoDB, without needing to use a Lambda function. There are existing AWS docs on using API Gateway as a proxy for DynamoDB, however, as usual, those only cover how to do this in the AWS console. In particular, I’ll show how I set this up using the Serverless Framework (or CloudFormation, as the bulk is really just CloudFormation code), and how you transform the web request’s JSON so it can be directly PUT into DynamoDB. Finally, I’ll talk about how to then do post-processing of the data via DynamoDB Streams.

Recently, I’ve begun to notice that imposter syndrome is not only for people that aren’t quote-unquote “qualified” to become Data Science because I’ve met people that have been studying math, computer science, stats, etc that are suffering from the same feat. In fact, I’ve spoken to people that work as data scientist that say they feel like they are faking it.