Articles on karannaoh's Blog

Final Week | Helm chart tweeks

karanpratapsingh43@gmail.com (karannaoh) — Thu, 22 Aug 2019 14:27:35 +0000

Added helm charts for Guillotina

Description

1 Deployement with 2 pods by default

1 configmap to mount guillotina configuration yaml on the pods

1 Service to expose guillotina

Guillotina allows user to pass arguments to override values defined in configuration fine similar to that user can pass arguments to `helm install` to override configuration.

Simple way to edit configurations is to edit configuration file in values.yaml

Week #11 Added validation for query Parameters

karanpratapsingh43@gmail.com (karannaoh) — Tue, 13 Aug 2019 18:05:04 +0000

PR:- #630

This Week:-

- Added OpenAPI definitions to @search service for both post and get request.

- Added validation of query parameters, so that a query parameters of a request can also be validated against defined schema only by adding `validate=True` in the service definition.

Stuck Anywhere:-

The way we are validating query params is iterating over all the params and validating there schema, it's short of like implementing our own jsonschema package which is a bit redundant and in efficient.

Week #10 | Helm chart Guillotina

karanpratapsingh43@gmail.com (karannaoh) — Wed, 07 Aug 2019 13:51:30 +0000

Issue:-#617

Task is to add helm charts for guillotina, so that guillotina can easily be deployed on kubernetes with all availble configurations.

allows mounting config file
allows overriding configuration values with G_ env variables, see: https://guillotina.readthedocs.io/en/latest/developer/commands.html?highlight=override#overridding-configuration

The helm chart should have support for configuring n number of databases and all the settings that guillotina supports. It would package these configuration settings into a config.json(or yaml if you prefer) that is mounted and referenced in the command arguments.

Make sure to provide the ability to also customize the image because people using this will likely be customizing the docker image used to package other things with guillotina(like customize addons).

This helm chart would serve as a generic chart to be able to deploy guillotina with.

Week #9 | Fixes in Swagger and schema validation

karanpratapsingh43@gmail.com (karannaoh) — Mon, 29 Jul 2019 07:29:13 +0000

PR #618

This week:-

Updated services parameters definitions according to OpenAPI 3.0

- Added path parameters for endpoints

- Added schemas to parameter definitions

- and added definitions to the endpoint which didnt had definitions of swagger

Week #8 | Fixes in decorator for validation

karanpratapsingh43@gmail.com (karannaoh) — Tue, 23 Jul 2019 06:15:53 +0000

Issue: #50 8 Validating all the request hiting Guillotina against proper jsonschema.

This Week:

PR: #517

Requests coming to the API were not being validated, task was to validate all the request hitting the API. Jsonschema is already defined for most of the endpoints, only a validation wrapper was required to be added to all the endpoint(along with jsonschema for missing endpoints and testcases for validation). Now all the request hitting being validated against a well defined schema and through 412(Precondition failed) along with proper error in case of a bad request.

Next Week:

Issue: #507

Added validation is a wrapper over all the endpoint causing too much redundancy in code base because its need to be added to all the endpoint. Next thing is to replace the validation with a decorator, which will validate the request automatically and will have very less code redundancy

Stuck anywhere:

Week #7 | Guillotina PubSub

karanpratapsingh43@gmail.com (karannaoh) — Mon, 15 Jul 2019 05:50:25 +0000

Commit :- 530f9c7e08151cf499d156c1fc756a0239bace5c

Worked on guillotina_pubsub which is a guillotina addon to subscribe to the content change via a websocket endpoint.

Last-Week:

Created a subscriber to listen to content change event and publish it on redis pubsub guillotina

This Week:-

Created websocket endpoint to @content-changes which a user can subscribe to in order to get the the changes from server about the content changed

Week #6 | Guillotina PubSub

karanpratapsingh43@gmail.com (karannaoh) — Mon, 15 Jul 2019 05:49:56 +0000

Issue #553

A consumer of the API needs to be able to subscribe to changes to a specific content object in Guillotina.

This week:-

Repository:- guillotina_pubsub

Created a new Addon for Guillotina which subscribe to changes made to any object and publish on a websocket service.

IObjectModifiedEvent is the event triggered everytime any object is modified, using this event details of changed object are publish on a channel of redis pubsub.

@content-changes is a websocket service which subscribe to all the changes made on the channel of redis pubsub and send to client.

Next week:-

Adding test cases for this Addon and necessary refractor to push the code to a new repository of guillotinaweb

Week #5 | Decorator for schema Validation

karanpratapsingh43@gmail.com (karannaoh) — Mon, 01 Jul 2019 06:04:31 +0000

This Week:

Issue: #507 Provide decorator for validating json schema request payloads.

Last week: added jsonschema validation to all the endpoints.

example- PR: #517

validator = get_schema_validator('Type_Of_schema') try: data = await request.json() validator.validate(data) behavior = data.get('behavior', None) return await delete_behavior(context, behavior) except jsonschema.exceptions.ValidationError as e: raise HTTPPreconditionFailed(content={ 'reason': 'json schema validation error', 'message': e.message, 'validator': e.validator, 'validator_value': e.validator_value, 'path': [i for i in e.path], 'schema_path': [i for i in e.schema_path], "schema": app_settings['json_schema_definitions']['Behavior'] })

problem in this:- This piece of code was added to all the endpoint causing too much code redundancy

Solution:- Adding an additional argument to the decorator of every route for the validation and validating request only if it's true.

example: - 1b8703dc28ff3487efa5ba09829af98b528740f4/guillotina/api/addons.py#L19

PR for this: - #544

Next Week:

Issue #553 Be able to subscribe to content changes.

Will be adding a new addon to guillotina which will subscribe to changes made on any object and publish it on a websocket endpoint.

WEEK #4 JSON schema validation

karanpratapsingh43@gmail.com (karannaoh) — Sun, 23 Jun 2019 10:12:18 +0000

Issue: #50 8 Validating all the request hiting Guillotina against proper jsonschema.

This Week:

PR: #517

Next Week:

Issue: #507

Stuck anywhere:

Week #3 Upgrading Guillotina_swagger

karanpratapsingh43@gmail.com (karannaoh) — Sun, 16 Jun 2019 17:58:06 +0000

Last week- Finished up adding authentication to Guillotina_swagger and updating security schemes for OpenAPI 3.

This Week :-

Issue: Authenticating the request send to baseurl.

This Week:-

Request send to baseurl needs to be authenticated in order to generate proper path object for swaggerUI. SwaggerUI provides `configs` in which authentication token can be added manually to every request sent.

Solution:- When a person tries to login, a new instance of swaggerUI object is created with the auth token in `configs`. github.

Problem in this solution:- New event is added to login button which is kind of a hacky way to fix things up, other than that logout button also requires a similar event handler to delete authentication token.

Other than this guillotina_swagger is completely upgraded to OpenAPI 3.

Next Week:-

Requests coming to the Api are not being validated, second task is to provide validation to the payloads of all the request hitting the API. JSON schema is already defined for most of the endpoints just adding the validation is required. Will be adding validation to all the endpoint and jsonschema for those where schema doesn't exist.

Stuck Somewhere:-

Still, the authentication of guillotina_swagger is working on a small hack and logout need to be fixed which needs to be fixed.

Week #2 Still Upgrading Guillotina_swagger :p

karanpratapsingh43@gmail.com (karannaoh) — Sat, 08 Jun 2019 10:50:16 +0000

Started to update Guillotina_swagger which is an addon to Guillotina, generates swagger UI for Guillotina, link to last week blog.

Things which were completed:- PR:- Upgrade swagger #3

Updated static js and css for OpenAPI 3. Issue#506
Updated swaggerObject(which is not OpenAPI object) according to OpenAPI 3 specifications. Issue#505

What did you do this week?:- Issue#513

Guillotina_swagger need authentication in order to generate the swagger object with the path object a user is authorized to access. Previously HTTP basic auth or JWT token were manually added to all the request headers and then the response object was rendered by swaggerUI which is now replaced by `securitySchemes` of OpenAPI 3. Instead of adding auth token to request header manually now `securitySchemes` of OpenAPI 3.

Definition of path objects is defined inline in the guillotina repository along with every endpoint. In OpenAPI 3 schema, parameters, responses, and definitions are now a part on a new object which is component, it's is one of the major changes required for the upgrade.

What is coming up next?

Path Object is created by Guillotina_swagger according to the permissions a user have, previously it was done by sending auth token to the baseurl. Now all the authentication is taken care by swaggerUI auth except the request send to get the OpenAPI object, which requires a fix.

Did you get stuck anywhere?

OpenAPI's authentication adds auth token to the request created to all the endpoints but the requests send to baseurl also need to have the auth token in order to generate the proper path object, it can be added manually to the request header but that would be a hacky solution. This problem still needs some research and brainstorming.

Week #1 | Upgrading Guillotina_swagger

karanpratapsingh43@gmail.com (karannaoh) — Fri, 31 May 2019 05:37:14 +0000

Issue #505

Guillotina_swagger is a addon to Guillotina which generates Swagger interface for the endpoints defined in Guillotina according to user access.

Task- Is to upgrade the Swagger object generated by Guillotina_swagger according to OpenAPI3 specs.

Terms which will be used in the article:-

Swagger/OpenAPI: Swagger is an open source framework used for documenting and testing APIs which has now upgraded to OpenAPI3

SwaggerUI: SwaggerUI is a tool provided by Swagger to generate a UI where all the functionalities of an API can be documented and tested.

Swagger Object/OpenAPI Object: Swagger Object is JSON or YAML where specifications about APIs are defined.

Guillotina_swagger generates swagger object(which is OpenAPI now) which is consumed by the frontend to create swaggerUI.

Changes required:-

Upgrading previously used swaggerUI packages(js,css files).
Upgrading Swagger Object according to new OpenAPI Specification.
Updating Javascript which is used to render swagger Object on the frontend

What did you do this week?

Started from understanding Guillotina and Guillotina_swagger and the things which need to be upgraded for OpenAPI3. Upgraded previously used SwaggerUI packages. Updated Swagger Object generated by Guillotina_swagger according to OpenAPI 3 specifications. Major changes which happened due to shift on OpenAPI 3 are -

swagger.host, swagger.basepath, swagger.schemes now comes under a new object swagger.servers
securityDefinitions, produces, consumes, definitions, parameter, responses comes under component which is also a new object in OpenAPI 3.

What is coming up next week?

Previously authentication was managed manually by adding authentication tokens inside the request created to get swagger object, this will be changed by using inbuilt securityDefinitions of SwaggerUI.
Schema, parameters, definitions, and responses are defined inside guillotina which are used inside path object of are needed to be updated as schema, parameters, definitions and responses are now a part of component object.

GSOC 2019 | karannaoh's blog | Guillotina

karanpratapsingh43@gmail.com (karannaoh) — Tue, 28 May 2019 17:52:13 +0000

Hello Everyone, It's Karan Pratap Singh an undergraduate student from India. I got selected in GSOC'19 project(Plone: Guillotina API Evolution) under Plone which is a sub-org of Python Software Foundation. I will be working on a major upgradation of Guillotina which is a fullstack Async framework for the next 12 weeks. My project involves

Upgrading Swagger2.0 documentation of the apis to Openapi 3.
Validating JSON schema to all the request and responses.
Working on adding all api's functionalities to the websocket endpoint to Guillotina.
Adding a mechanism to version the API.
And adding helm chart for easy deployment of Guillotina

I will be writing weekly blogs for the next 12 weeks explaining my work and progress.

Thank you for reading