I’ve been using Google Cloud since the yearly days, I think it was year 2009 when I’ve deployed my first website to the App Engine. Since then I was doing consulting, made many projects, experiments, and so on. For past few months I’ve been working on TipTop.io, it’s a SaaS solution for analytics and monitoring for applications hosted on Google Cloud Platform.
The project in the early stage currently, and it supports only Google App Engine at this moment. But support for Cloud Containers/GKE is coming soon (and probably plain Kubernetes in near future).
Google Cloud have a nice Log Service, with some cool features (like Traces, I wrote before), but it lacks real analytics based on this logs. Like Kibana.
Fortunately Google Cloud can export logs to Cloud Storage. What’s cool is that this logs are in JSON format, so we could easily import them into ELK, without any complex Logstash configuration (honestly I cannot say that JSON schema fits well ELK, but still it’s easy to import).
I’ve prepared a basic Docker container with Elasticsearch, Logstash and Kibana configured for Appengine logs. Run ELK container with:
(you can get sources of this Docker image there).
Fixing webapp speed is really hard job, mostly because it’s hard to find a bottleneck. And I want to show some tools that Google Appengine gives you for this job. Actually i’m going to tell about combination of two tools, that works just perfectly together.
First thing is Traces (under Monitoring tab in Cloud Console). It’s kind of a new tool, and I didn’t pay much attention earlier, just played a little. I thought it’s just another view to your logs, from Appengine APIs point of view:
It shows you details about requests, with information about which API calls were made, how much time server spent on them, how much it did cost, etc. Pretty useful information btw.