DoiT-Easily

Written with Dave Cavaletto

Developing a solution to sell on Google Marketplace is not technically difficult. The main challenge is that this is an enterprise-style integration, in which you are connecting to the business systems of a second company, Google; as opposed to accessing freely available APIs as in Google Cloud Platform. Google must approve your GCP project before you create your integration, and then approve your solution once it is ready for release.

DoiT-Easily is an open-source project that shows a fully functional sample integration created by DoiT engineers. The integration works, but it is not a production-ready product, nor is it a supported project. Rather, it is useful as a quickstart example from which to learn and potentially adapt into a production solution.

Bring your product to market, faster!

In an earlier post, we gave a high-level overview of the creation of a Marketplace offering. This article is a more detailed run-through of DoiT-Easily, describing key architectural elements as well as challenges you will experience in development. In this article, we include features that DoiT-Easily implements that are not a necessary part of a Marketplace integration, and also features found in some Marketplace integrations that are not present in DoiT-Easily. For more details, please see the documentation and code in the DoiT-Easily Github project, along with public Marketplace documentation from Google.

The following diagrams break down the two main flows: Creating a new user and creating a new entitlement (purchase). The new-entitlement flow is the same as the flow for changing or canceling an entitlement. For new users who are also signing up for an entitlement, both flows are triggered in parallel.

After the flow diagrams, please see a diagram and discussion of components.

New User Flow

Entitlement Flow

This architecture diagram below from DoiT-Easily, we see the components of your integration and the Google APIs that it integrates with.

Your functional components are in the lower half of the diagram, while Google’s APIs are in the top half.

Your SaaS, the one that you are integrating into Google, is not present in this example. Your SaaS servers could be deployed to another Google project, or the same one as the integration system.

Below, we go into detail on these components.

DoiT-Easily Architecture

Frontend

The frontend integration in the diagram is a web app, including client and server. DoiT-Easily implements an HTML/AJAX client with HTML templates. The server is in Python and deployed to Cloud Run. Of course, you can choose other technologies in your own implementation.

The front-end client presents the landing page that subscribers are redirected to during the sign-up flow, after GCP Marketplace forwards them. The front-end checks the JWT in the request, to authenticate that it is from Google. After the user signs up, the front-end server notifies Google’s Procurement API, so creating the subscriber’s procurement account. The entirety of the Frontend Integration’s server logic is contained in the login() function, which can be reached at the /activate or /login path.

Backend

The backend integration is a server that is also deployed to Cloud Run. In DoiT-Easily, this is the same Cloud Run service, though you might decide to create separate front-end and back-end services in your application. The backend receives Pub/Sub messages from Google which notify about Entitlement. (“Entitlement” is Google’s term for a subscription to your offering.)

Note that the api.py file at the heart of the Cloud Run application has a lot of web endpoints that are rarely used, so focus on the main flows. The main flow is triggered via Pub/Sub through a push-subscription handled by the handle_subscription_message() function, which can be reached at the /v1/notification path.

Backend-UI

DoiT-Easily offers an additional UI for the backend, one that is not required by the basic Marketplace integration architecture. It can be used to list pending Entitlements and manually approve those entitlements, if your system is not designed to automatically approve them. The server code can be viewed at the /app path in app.py and is secured by Identity-Aware Proxy (IAP).

In app.py , you will see several endpoints supporting this app including the app’s main page; as well as methods to list, approve, or reject entitlements and list accounts.

Other Endpoints

All other endpoints in th e api.py file are available for use as an authenticated proxy to the Procurement API, so that a custom solution can interact with DoiT-Easily rather than the Procurement API directly.

Usage Reporter

The Usage Reporter, not part of DoiT-Easily, reports the customer’s usage of your product to Google’s Service Control API. This is needed if billing is by resources, like the amount of data processed, number of calls to your solution, etc. However, if you offer a solution that is free or paid for by monthly subscription, then this component is not needed, which is why DoiT-Easily does not include it.

If you implement a Usage Reporter, you will likely want to implement your own datastore to track entitlements (subscribers to your offering), and use these records to drive usage reporting to Google. As-is, DoiT-Easily does not implement a datastore, instead relying on the Procurement API to store all Entitlements and their associated state.

ISV Provision Service

If each new customer imposes new resource requirements on your SaaS solution, you can create this component to manage these resources. For example, if each customer gets its own pod or namespace in Google Kubernetes Engine, the ISV Provision Service would set that up. Because this service is not a necessary part of most SaaS solutions, DoiT-Easily does not include it.

Other Cloud Components

Besides the client and server components in the diagram, DoiT-Easily includes the following cloud infrastructure resources. Terraform modules in DoiT-Easily will deploy these for you unless otherwise mentioned below.

Specified by Google Marketplace

• IAM: This includes permissions to your project for several Google service accounts. The permissions for your service account to access Google are not defined in this step, but rather requested through the Producer Portal (see Step 5 in “Set up”, documented here.)
• Pub/Sub: A topic on which Google sends you messages about entitlement events. The topic is in a Google-owned project and is created by Google when you go through the “Set up” described below. Your backend subscribes to this topic.

Other Cloud Services

DoiT-Easily uses these cloud technologies; you can choose others, so long as they provide the same functionality.

• Cloud Run services for front-end and back-end, as mentioned above.
• Cloud DNS Zone: Each Marketplace listing has its own public address, to which the Google Marketplace forwards sign-up requests. The DNS Zone serves these domains. The Terraform does not create this resource; we assume you have an existing zone.
• A Load Balancer receives the requests, including those forwarded from Google Marketplace, and routes them to the service for the relevant Marketplace listing. The Load Balancer has the associated public IP address, TLS certificate, IAP, and routing rules.
• A configuration file for Cloud Run, stored as a Secret in GCO Secret Manager. The Terraform creates the Secret object. When deploying DoiT Easily, be sure to update the value in Secret Manager as needed. This increments the version number, which should be updated in the tfvars. The configuration contains some environment values that need to be kept secret, and for convenience, also some that do not.

Setting up DoiT-Easily has the following phases. Several of them have Terraform modules. Copy file example.tfvars to terraform.tfvars and edit the values, then run terraform init and terraform apply.

Pre-requisites

The pre-requisites phase is called that because it comes before Google approves your project. The Terraform creates the project itself and IAM roles for Google’s service accounts to your project, as well as a role for the user who is running the deployment.

Set up with Google

You set up your project Google by submitting a form to request approval, which can take several weeks to come through. (Ask DoiT for help in navigating this!) After approval, you fully define your listing in the Producer Portal, including pricing, production information, and the service account to which Google will provide permissions.

Deployment

You build the Cloud Run app using Cloud Build, pushing the image to Artifact Registry. Then you deploy it with Terraform, along with the cloud resources it needs like DNS Zone and Load Balancer.

Testing

During your development process, you can run DoiT-Easily locally, and run the included unit tests.

Once the solution is deployed, you can test it manually, stepping through the user registration, purchase, and approvals in your Marketplace project. (Until the listing has been published, it is in a private preview mode.)

An automated integration testing framework from Google is also available; Google requires that it passes before publishing.

If you are only offering private offers, the automated testing will never succeed. You can skip this step and still publish your listing.

For the final step of publishin g, you ask for manual approval from Google. This means that you cannot make frequent fixes and improvements by pushing to production several times a day, as you would with continuous deployment. Each new version may have a round-trip of several days while awaiting approval.

Any implementation of Marketplace integration, including DoiT-Easily, has some essential limitations, as contrasted with typical cloud applications. There are also some possible features that are not implemented by DoiT-Easily.

Approval Needed Per Project

After you set up the pre-requisites, Google needs to approve your project. This means that you will have only one project where your solution can work; you will not have separate dev, testing, staging, and production projects.

The resources that must be approved before your project can be used cannot be easily deleted and re-created. This means that you cannot have an end-to-end test, one in which a script creates a new sandbox project, sets up the application, runs tests, and then deletes everything. Likewise, it is not possible to clean up all resources in your Marketplace project and start afresh. You can run a test using the built-in testing and preview facilities in Marketplace, but this test will not start and end with a clean slate.

Similarly, setting up separate new dev environments for each development team member is not possible. You can instead share a project between team members. However, you will have to design an architecture in which team members create separate listings; they will share all other infrastructure, such as the project, the IAM settings, the Load Balancer, the Pub/Sub topic, and the DNS zone. Each app will need to subscribe to the Pub/Sub topic, and then only process the messages pertaining to it. DoiT-Easily has not fully implemented multiple-listing capacity: Though it does not stop you from creating multiple listings, the Terraform deployment code is not built for this.

This also means that to run DoiT-Easily, like any other Marketplace integration, you will need an official Marketplace project that will go through Google approval.

Alternative Scenarios

DoiT-Easily was created to present one scenario, but as usual with enterprise-style business-to-business integration, Marketplace solutions often are built in custom variations. One example is a private-offer-only solution; this gives vendors more control over their sales process and saves the need for a front-end as described above. However, this necessitates an internal tool to manage the private offers, and many vendors build an internal web app for this purpose.