Run a sample application with Dapr with OpenShift

Dapr web site - link

Summary: Dapr: 'An event-driven, portable runtime for building micro-services on cloud and edge'

The point of this page is to demonstrate that the framework can be tried on Openshift as well as minikube, which is documented as the normal way to run the sample code. Please note that the security changes shown are not recommended other than for a proof of concept and that more fine grained permissions should to be set.

How to run a sample application on OpenShift

The following instructions take the sample (2. Hello-Kubernetes) tutorial and get it running on OpenShift. This sample is located: https://github.com/dapr/samples/tree/master/2.hello-kubernetes

Pre-reqs:

• crc installed locally from RedHat (need a developer account to download from cloud.redhat.com) 
• dapr installed locally (see dapr site for download instructions) -

Changes from sample documented instructions:

1. Not sure this is necessary, but I used a different Redis install from here: https://www.callicoder.com/deploy-multi-container-go-redis-app-kubernetes/ this was because I assume I couldn't get the configuration of the password correct and security for Redis is not a priority to just 'kick the tyres'
2. Run crc  i.e. 'crc start' ensuring it was enough CPU & RAM assigned e.g. on MacBook with 16GB of RAM my config is: 7 cpus, 16384Mb RAM

3.     Login to OpenShift as user kubeadmin & create test project test1: 'oc new-project test1'

4.     Add the following permissions to the project:

o   oc adm policy add-scc-to-user anyuid -z default -n test1

o   oc adm policy add-scc-to-user privileged -z default -n test1

5. Deploy the redis-master.yaml from the directory
      go-redis-kubernetes/deployments directory: 'oc apply -f redis-master.yaml'
6. Create a file called redis-state.yaml, and paste the following:

apiVersion: dapr.io/v1alpha1

kind: Component
metadata:
  name: statestore
spec:
  type: state.redis
  metadata:
  - name: redisHost
    value: redis-master:6379 

7.  Create a file called redis-pubsub.yaml, and paste the following:

apiVersion: dapr.io/v1alpha1

kind: Component

metadata:

  name: messagebus

spec:

  type: pubsub.redis

  metadata:

  - name: redisHost

    value: redis-master:6379 

8. Deploy dapr: for crc only the advanced helm deployment worked:

oc adm policy add-cluster-role-to-user cluster-admin system:serviceaccount:test1:default

oc adm policy add-cluster-role-to-user cluster-admin system:serviceaccount:kube-system:default

helm init

helm repo add dapr https://daprio.azurecr.io/helm/v1/repo

helm repo update

helm install dapr/dapr --name dapr --namespace test1

Note: it may take a few minutes for tiller to be installed (assuming tiller still there) after helm init

If you get the error 'components.dapr.io' already exists then do 'helm delete --purge dapr' 

9.  Add the following permission to the dapr service account:

·       oc adm policy add-cluster-role-to-user cluster-admin system:serviceaccount:test1:dapr-operator

10. Check that the 3 dapr pods are running ok with no errors logged

11.  Apply both of the above files created in steps (6) and (7) to OpenShift using 'oc apply -f <file>'

12. Change the 2.Hello-Kubernetes redis deployment file in the deply directory, redis.yaml metadata part to just contain the following two lines i.e. remove the password part:

-name redisHost

 value: redis-master:6379

13. Deploy all the files in the deploy directory: redis.yaml, node.yaml & python.yaml
14.  When looking at the nodeapp logs orders should be seen and being seen as persisted.

Run the Binding (Kafka) Sample

Assuming the first example has been performed and crc, helm and dapr already installed.

1. Install Kafka using the Strimzi operator - see
2. Run: oc apply -f https://github.com/strimzi/strimzi-kafka-operator/releases/download/0.14.0/strimzi-cluster-operator-0.14.0.yaml -n test1
3. Download the file:  https://raw.githubusercontent.com/strimzi/strimzi-kafka-operator/0.14.0/examples/kafka/kafka-persistent-single.yaml e.g. 'curl -O' or 'wget'  & edit to make the 100Gi volumes 1Gi volumes
4. Then oc apply -f kafka-persistent-single.yaml -n test1
5. oc adm policy add-cluster-role-to-user cluster-admin system:serviceaccount:test1:strimzi-cluster-operator
6. Create the kafka topic:

oc -n test1  run kafka-producer -ti --image=strimzi/kafka:0.14.0-kafka-2.3.0 --rm=true --restart=Never -- bin/kafka-console-producer.sh --broker-list my-cluster-kafka-bootstrap:9092 --topic sample

6. In the dapr/samples/5. bindings/deploy file change the kafka_bindings.yaml file and change the value 'dapr-kakfa.kafka:9092' string to 'my-cluster-kafka-bootstrap:9092' to align with the Kafka pod created by the Strimzi operator
7. Apply kafka_bindings.yaml & node.yaml & python.yaml using 'co apply -f <file>'
8. The logs from Node should the example working

An easy way to build a RESTful micro service in Go to access a Cassandra Table

TL;DR

This code repo allows one to run a single program to create a RESTful micro service that can read from a Cassandra Table a single row by using the primary key fields to restrict the SELECT clause and return it in JSON format. As of 22nd April 2019 it can also POST into simple tables!

It is hosted by the go-swagger framework and uses gocql as the Cassandra data access layer. The code is all Go and a single main programme provides a single command to accomplish this feat. It should be considered alpha software because I have only tested it using a limited number of use cases, but I hope it should suffice for many uses as is.

Motivation

I had a few reasons to write this code. One was a question from a former client "Can't we generate the micro services to read data from Cassandra?", another was a desire to learn a new programming language, Go, with a real project, but probably the main reason was I wanted to create something other than PowerPoints like I did in the good old days 😏 

My Approach

I wanted to generate all the code so I started looking at the Swagger tooling, but whilst looking good for Java, I didn't find that the generator for Go worked very well. A little looking around and I came across go-swagger, which worked first time. This tool lets one generate server or client side code from an input swagger specification and there are loads of capabilities it provides that I haven't even looked into.

The data access options were quite limited, but once I found gocql  and checked out the code I was happy with my choice. 

Both of the core technologies have active communities and mature code.

That left the question of how to create the handler code to run in the framework created by go-swagger that would retrieve the data. To do this I needed to write some code myself. The approach I took was a 4 step process:

1. Parse the Cassandra DDL that defined the table and any required types (UDTs) to create the swagger API specification for the table
2. Use go-swagger to generate the RESTful framework for the API defined in the created swagger file
3. Using the parser output from (1) create the handler function that uses gocql to access the Cassandra table
4. Patch the generated go-swagger code to call the functions of my generated data access code

All of the above can (and has) been implemented in a single main.go file

Code Structure & Running Output

The main.go program resides in the root folder, the sub directories contain:

• handler - the code that generates the data access code
• parser - the code that parses the Cassandra DDL file 
• swagger - the code that takes the parser output and creates the swagger output file

The main.go file expects a number of program arguments to be set:

• -file 
• -goPackageName
• -dirToGenerateIn

The file parameter defines the full path of the input Cassandra DDL file to process.

The goPackageName is that of the desired Go package name.

The dirToGenerateIn is optional as it defaults to /tmp, but without go modules needs to be set to a file under $GOPATH/src

The example command I used was:

go run main.go -file=/Users/stevef/Source_Code/go/src/github.com/stevef1uk/test4/t.cql\  -goPackageName=github.com/stevef1uk/test4 \

- dirToGenerateIn=/Users/stevef/Source_Code/go/src/github.com/stevef1uk/test4

There are several other flags that may be set e.g. -debug=true & -endPoint=<end point name> 

Then, in the directory where go-swagger will have created the framework run:

export CASSANDRA_SERVICE_HOST=127.0.0.1

go run cmd/simple-server/main.go --port=5000

Assuming all is well you should see:

2019/02/19 22:18:55 Tring to connect to Cassandra database using  127.0.0.1

2019/02/19 22:18:55 Yay! Connection to Cannandra established

2019/02/19 22:18:55 Serving simple at http://127.0.0.1:5000

At this point the microservice will be accessible at the following URL:

http://127.0.0.1:5000/v1/<table name>

note:  <end point name> would overide <table name> 

e.g. curl -X GET "http://127.0.0.1:5000/v1/accounts4?id=1&name=steve&time1=2013-01-01T00:00:00.000Z"

The Parser -

In the (very distant) past I had used the Unix tools, lex & yacc to create and parse a formal language for the pretentiously named General Purpose Test Tool. This was a Visual Basic like language that I designed to let a software house test its 'C' code in the relatively dark ages of computing (early 1980s). I even had a nice book on these tools, which my wife persuaded me to throw away a year or so ago when she she pointed out that this book had been in our attic for decades and I was not likely to need it again 😢

I took a look at the Cassandra schema for DDL and decided parsing it using a lex & yacc approach was going to be very challenging, so I decided to try using regular expression matching instead. This approach worked, but the code was very hard to follow making it hard to extend and maintain. Therefore, I rewrote it using the Finite State Machine approach together with regular expressions. I did look at a couple of FSM libraries, but in the end wrote my own for this.

Go-Swagger

Once a swagger file is created the command 'swagger generate server -f swagger.json' will create the server code. This command will also list the packages that need to be installed to build the framework successfully.

The file in the restapi folder called configure_simple.go is the one that step (4) above patches. Step (3) creates the generated data access file in a new directory called data. This file is called Generatedhandler.go

Handler Options

By default all of the fields defined as primary fields will be used to select data from the database, if the flag numberOfPrimaryKeys is set to a number then only this number of keys will be used in the select statement in the order in which they are defined. All will still need to be passed on the RESTful API call though.

The consistency flag can be defined to one of the standard gocql consistency modes, the default is gocql.One

The allowFiltering flag was supposed to add this clause to the generated select statement, but I seem to have forgotten to implement it 😑

The handler code has its own Setup() and Stop() functions that establish a connection to the Cassandra database defined by the CASSANDRA_SERVICE_HOST environment variable.

The micro service can be scaled horizontally. If deploying the micro service to something like OpenShift or Kubernetes, I found the hard way that the pod needs to be configured to listen on all networks e.g. --host=0.0.0.0 in order to run successfully.

Tests

I like testing my software as it gives me confidence to refactor so there are test modules in each sub package that contain tests.

The hard part with testing the handler code was having to define Cassandra schemas, populate the resultant tables with data! I have tested some User Defined Types alongside a table.

The most complex test I ran had the following schema:

CREATE TYPE demo.simple (

       dummy text

    );

    CREATE TYPE demo.city (

    id int,

    citycode text,

    cityname text,

    test_int int,

    lastUpdatedAt TIMESTAMP,

    myfloat float,

    events set<int>,

    mymap  map<text, text>,

    address_list set<frozen<simple>>

);

CREATE TABLE demo.employee (

    id int,

    address_set set<frozen<city>>,

    my_List list<frozen<simple>>,

    name text,

    mediate TIMESTAMP,

    second_ts TIMESTAMP,

    tevents set<int>,

    tmylist list<float>,

    tmymap  map<text, text>,

   PRIMARY KEY (id, mediate, second_ts )

 ) WITH CLUSTERING ORDER BY (mediate ASC, second_ts ASC)

Note: using describe table is the best way to populate this file. The FSM is configured to look for the WITH text to end processing. If you have a data type or field with the name WITH then you will need to change this to WITH CLUSTERING.

To insert data into this table I used:

insert into employee ( id, mediate, second_ts, name,  my_list, address_set  ) values (1, '2018-02-17T13:01:05.000Z', '1999-12-01T23:21:59.123Z', 'steve', [{dummy:'fred'}], {{id:1, mymap:{'a':'fred'}, citycode:'Peef',lastupdatedat:'2019-02-18T14:02:06.000Z',address_list:{{dummy:'foobar'}},events:{1,2,3} }} ) ;

Then, running the main & testing gave me:

curl -X GET "http://127.0.0.1:5000/v1/employee?id=1&mediate=2018-02-17T13:01:05.000Z&second_ts=1999-12-01T23:21:59.123Z"

[{"address_set":[{"address_list":[{"dummy":"foobar"}],"citycode":"Peef","events":[1,2,3],"id":1,"lastupdatedat":"2019-02-18 14:02:06 +0000 UTC","mymap":{"a":"fred"}}],"id":1,"mediate":"2018-02-17 13:01:05 +0000 UTC","my_list":[{"dummy":"fred"}],"name":"steve","second_ts":"1999-12-01 23:21:59.123 +0000 UTC","tevents":[],"tmylist":[],"tmymap":{}}]