Beginner’s Guide to Deploying NLP Model to AWS Serverless Architecture, WSL2, and Windows10

This is a beginner’s guide to deploying an NLP Model to AWS Serverless Architecture. This is something I learned from this course. I took the course and used to build and deploy using WSL2 and Windows.

Step

Open a remote folder or workspace using steps defined here.

Create Virtual Environment Using Conda

conda create -n spacy-serverless python=3.6 pylint rope jupyter

Activate spacy-serverless environment using

conda activate spacy-serverless

Use pip to install spacy to match the similar process between docker and serverless.

There are often multiple versions of python interpreters and pip versions present. Using python -m pip install <library-name> instead of pip install <library-name> will ensure that the library gets installed into the default python interpreter.

python -m pip install spacy==2.1.3

Install English Language models using

python -m spacy download en_core_web_sm

It contains an English pipeline optimized for CPU. Components: tok2vec, tagger, parser, senter, ner, attribute_ruler, lemmatizer.

You can check whether language models that you have downloaded are compatible with the currently installed version of spaCy by running validate command. The command is also useful to detect out-of-sync model links. 

python -m spacy validate

Try out the model in Jupyter Notebook and installation of Spacy

import spacy
from spacy.lang.en.examples import sentences 

nlp = spacy.load("en_core_web_sm")
doc = nlp(sentences[0])
print(doc.text)
for token in doc:
    print(token.text, token.pos_, token.dep_)
spaCy Example

Create Serverless Project

sls create --template aws-python --name ner-api

You create a directory called my-serverless-project and create a project using sls create. Template with -t aws-python3. is also specified. Serverless comes bundled with several templates that set some sensible defaults for you in serverless.yml. In this case, you are also specifying the AWS template for Python 3.6. If your project is Python 2.7, use aws-python2.

The -n my-serverless-project specifies a service name, and you can change this to whatever you want to name your project.

Install serverless-python-requirements

sls plugin install -n serverless-python-requirements@4.2.4

Make changes to handler.py file

print('container start')

# reduces the python deployment packages
try:
    import unzip_requirements 
except ImportError as error:
    pass

print('unzipped')

import json
import en_core_web_sm

MODEL = en_core_web_sm.load()
print('model loaded')

def create_ner_spans(text):

    doc = MODEL(text)
    spans = []
    for ent in doc.ents:
        span = {
            'start': ent.start_char,
            'end': ent.end_char,
            'type': ent.label_
        }
        spans.append(span)
    return spans


def handle_request(event, context):
    text = event['body']
    print(text)
    spans = []
    if text is not None:
        spans = create_ner_spans(text)
    print(spans)

    body = {
        'spans': spans
    }

    response = {
        "statusCode": 200,
        "body": json.dumps(body),
        'headers': {
            'Content-Type': 'application/json',
            'Access-Control-Allow-Origin': '*'
        }
    }

    return response

Amend serverless.yml file


service: ner-api # NOTE: update this with your service name

provider:
  name: aws
  runtime: python3.6
  stage: dev
  region: us-east-1

# you can add packaging information here
package:
#  include:
#    - include-me.py
#    - include-me-dir/**
  exclude:
    - node_modules/**
    - .vscode/**
    - __pycache__/**
    - .ipynb_checkpoints/**
    - (*).ipynb
    - .empty/**

functions:
  recognize-named-entities:
    handler: handler.handle_request
    memorySize: 1536
    timeout: 30
    events:
      - http:
          path: ner
          method: post

custom:
  pythonRequirements:
    dockerizePip: true 
    slim: true # reduce deployment package size
    zip: true
    noDeploy: [] # emits python requirements already installed such as boto3
    useDownloadCache: true # caches download and speeds up deployment
    useStaticCache: true # caches output of pip

plugins:
  - serverless-python-requirements
Dealing with Lambda’s size limitations

To help deal with potentially large dependencies (for example: numpyscipy and scikit-learn) there is support for compressing the libraries. This does require a minor change to your code to decompress them. To enable this add the following to your serverless.yml:

custom:
  pythonRequirements:
    zip: true

and add this to your handler module before any code that imports your deps:

try:
  import unzip_requirements
except ImportError:
  pass

Create requirements.txt file

spacy==2.1.3
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.1.0/en_core_web_sm-2.1.0.tar.gz

Create event.json file

{
    "body": "Wikipedia is a free online encyclopedia, created and edited by volunteers around the world and hosted by the Wikimedia Foundation."
}

Invoke local serverless function to test

sls invoke local --function recognize-named-entities --path event.json

Deploy using the following command

export AWS_ACCESS_KEY_ID=<your-key-here>
export AWS_SECRET_ACCESS_KEY=<your-secret-key-here>
# AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are now available for serverless to use
sls deploy

Invoke function globally and log output

sls invoke --function recognize-named-entities --path event.json --log

References:

  • https://blog.newrelic.com/engineering/create-a-serverless-function-in-python/
  • https://www.serverless.com/blog/flask-python-rest-api-serverless-lambda-dynamodb#using-the-quick-start-template
  • https://stackoverflow.com/questions/40071125/serverless-framework-python-and-requirements-txt
  • https://www.serverless.com/plugins/serverless-python-requirements
  • https://www.udemy.com/course/deploy-serverless-machine-learning-models-to-aws-lambda/

Leave a Reply