Good blog post by the engineering team at Stripe on using Kubernetes to run a distributed cron scheduler
Good blog post by Timothy Downs on how queues and data streams work with a layman example at https://hackernoon.com/introduction-to-redis-streams-133f1c375cd3
Quoting the example here
We have a very long book which we would like many people to read. Some can read during their lunch hour, some read on Monday nights, others take it home for the weekend. The book is so long that at any point in time, we have hundreds of people reading it.
Readers of our book need to keep track of where they are up to in our book, so they keep track of their location by putting a bookmark in the book. Some readers read very slow, leaving their bookmark close to the beginning. Other readers give up halfway, leaving theirs in the middle and never coming back to it.
To make matters even worse, we are adding pages to this book every day. Nobody can actually finish this book.
Eventually our book fills up with bookmarks, until finally one day it is too heavy to carry and nobody can read it any more.
A very clever person then decided that readers should not be allowed to place bookmarks inside the book, and must instead write down the page they are up to on their diary.
This is the design of Apache Kafka, and it is a very resilient design. Readers are often not responsible citizens and often will not clean up after themselves, and the book may be the log of all the important events that happen in our company.
When you do your job well, no one notices. When you screw up the whole internet notices – Julia Grace
during a great keynote speech at Velocity 2017 on how to build efficient engineering teams.
Mark Horstman, on his podcast about when not to use email (https://www.manager-tools.com/2017/05/when-not-use-email-part-2)
No delivery of information is purely about information. Every delivery of information has some effect on the relationship that is formed during the communication or (relationship) that previously existed.
“We in this country are facing a lack of visionary servant leadership. Any leader you can think of will say they are fighting for people, and this is a necessary but insufficient condition for being a leader, to fight for people that need you. But what we really need for real vision is level two and level three servant leadership. What’s level two servant leadership? It’s fighting for people that need you that you don’t need. Level three servant leadership is fighting for people who don’t like you. This is the problem, where we split into tribes where leaders only lead their followers.” Arthur Brooks, February 18, 2016, TED 2016, Vancouver, British Columbia
Miki Agrawal on perseverence
It takes 10 years to be an overnight success
Quote from a talk by Herschel Walker at Copart’s annual Global Leadership conference
“To get there you have to work
To stay there you still have to work”
Over the last week, I moved this blog from a LAMP (Linux, Apache, MySQL, PHP) stack to LEMP (Linux, Nginx, MySQL, PHP) stack. Have a blog post in the works with all the gory details, but wanted to quick document a quirk in the WordPress + Nginx combination that broke permalinks on this site.
Permalinks are user friendly permanent static URLs for a blog post. So for example this particular blog post’ URL is
This works by default in Apache because WordPress puts in the required rewrite rules.
To get it work in Nginx, you have to add the following config in the Nginx site configuration
Under the / location context, add the following
try_files $uri $uri/ /index.php?$args;
This is essentially telling Nginx to try to display the URI as is, and if it fails that, pass the URI as an argument to index.php.
We ran into an issue at work recently, when trying to deploy a spring boot based micro service. The application included some libraries that were downloaded by the application using maven during runtime.
The environment that this application was running in had controlled access to the Internet. We configured the jvm parameters using the following parameters
-Dhttp.useProxy=true -Dhttp.proxyHost=PROXY_SERVER_NAME -Dhttp.proxyPort=xxxx
when running the application, we were getting failed dependency errors that stated something like this
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.xxxxx.boot:xxxx -boot-autoconfigure:pom:0.0.1-SNAPSHOT from/to central (http://repo1.maven.org/maven2/): Connection refused at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246) at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:223) at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:334)
Lot of head scratching, because the server running the application had connection to the proxy server and the JVM parameters were accurate.
Finally, we figured out Maven doesn’t use the JVM startup parameters and had to have proxy access configured specifically. There are two ways to do this
set MAVEN_OPTS= -Dhttp.proxyHost=PROXY_SERVER_NAME -Dhttp.proxyPort=xxxx
<proxies> <proxy> <id>genproxy</id> <active>true</active> <protocol>http</protocol> <!--username>proxyuser</username--> <!--password>proxypass</password--> <host>myproxy.mycompany.com</host> <port>8080</port> <nonProxyHosts>*.mycompany.com|127.0.0.1</nonProxyHosts> </proxy> </proxies>