Web

Collection of tools to serve local content on your workstation to the Internet

A quick collection of tools you can use to serve/publish content/applications on your local dev to the Interwebs. Some use cases for these types of tools..

  • Developed a static website and want to show it someone that is not right next to you.
  • Developed an API that you want an app or user to access from the web

List of tools:

  • https://ngrok.com/ : Most popular tool for this purpose. the free tier is enough for most use cases.
  • https://tunnelto.dev/ : Latest entrant in this space. In addition to have a paid hosted service, you can run this for free on your own server. But that defeats the pupose of having a tool to use in a pinch to share content :).
  • http://pagekite.net/ : Been around for 10+ years. Similar to tunnelto.dev, you can run this on you own server or pay (very nominal price) for the hosted service.

June 25, 2020 by Technology Web 1

HOW TO : Configure nginx to use URI for modifying response content

That was a pretty long title for the post :). I love nginx for it’s flexibility and ease of use. It is like a swiss army knife.. can do a lot of things :).

We needed to serve some dynamic content for one of our use cases. If user visits a site using the following URL format http://example.com/23456789/678543 , we want to respond with some html content that is customized using the 23456789 and 678543 strings.

A picture might help here

Here’s how this was achieved

  • Define a location section in the nginx config to respond to the URL path specified and direct it to substitute content 
    location ~ "^/(?<param1>[0-9]{8})/(?<param2>[0-9]{6})" {

            root /var/www/html/test/;
            index template.html;
            sub_filter_once off;
            sub_filter '_first_param_' '$param1';
            sub_filter '_second_param_' '$param2';
            rewrite ^.*$ /template.html break;
    }

create a file named template.html with the following content in /var/www/html/test

Breaking down the config one line at a time

location ~ "^/(?<param1>[0-9]{8})/(?<param2>[0-9]{6})" : The regex is essentially matching for the first set of digits after the / and adding that as the value for variable $param1. The first match is a series of 8 digits with each digit in the range 0-9. The second match is for a series of 6 digits with each digit in the range 0-9 and it will be added as the value for variable $param2

root /var/www/html/test/; : Specifying the root location for the location.

index template.html; : Specifying the home page for the location.

sub_filter_once off; : Specify to the sub_filter module to not stop after the first match for replacing response content. By default it processes the first match and stops.

sub_filter 'first_param' '$param1'; : Direct the sub_filter module to replace any text matching first_param in the response html with value in variable $param1.

sub_filter 'second_param' '$param2'; : Direct the sub_filter module to replace any text matching second_param in the response html with value in variable $param1.

rewrite ^.*$ /template.html break; : Specify nginx to server template.html regardless of the URI specified.

Big thanks to Igor for help with the configs!!

May 7, 2020 by HOWTO Technology Web 1

Why ADP?

ADP is a $70B+ (by market cap as of August 2019) company and yet cannot get a simple redirect correct. If someone that is asked to use it’s employee performance management system types in tms.adp.com (like most people would do), they get this nice friendly error

If by some magical and mystical reason, they type in https://tms.adp.com, they get this login page

I find it mind boggling that such a mature company cannot figure out

  1. Customer experience
  2. 301/302 http redirects
  3. HTTP Strict Transport Security (HSTS)

End Rant and sorry to all my friends that work at ADP 🙂

August 14, 2019 by Rantings Technology Web 0

Optimizing cache infrastructure

I love when engineering teams share their tricks of trade for other organizations to benefit. While this might seem counter-intuitive, sharing knowledge makes the entire ecosystem better.

Etsy‘ engineering team does a great job of publishing their architecture, methodologies and code at https://codeascraft.com.

This particular article on how they optimize their caching infrastructure (https://codeascraft.com/2017/11/30/how-etsy-caches/) is pretty enlightening. I always thought the best method to load balance objects (app hits, cache requests, queues etc) to hosts was to use mod operations. In this blog post Etsy’ team talk about using consistent hashing instead of modulo hashing.

At a high level, it allows cache nodes to fail and not impact the overall performance of the application drastically in addition to making it easy to scale the number of nodes. This method is useful when you have a large amount of cache nodes.

More reference links

  • http://www.tom-e-white.com/2007/11/consistent-hashing.html
  • https://www.toptal.com/big-data/consistent-hashing
  • https://en.wikipedia.org/wiki/Consistent_hashing

 

January 3, 2018 by Programming Technology Web 0

HOW TO : Configure nginx for WordPress permalinks

Over the last week, I moved this blog from a LAMP (Linux, Apache, MySQL, PHP) stack to LEMP (Linux, Nginx, MySQL, PHP) stack. Have a blog post in the works with all the gory details, but wanted to quick document a quirk in the WordPress + Nginx combination that broke permalinks on this site.

Permalinks are user friendly permanent static URLs for a blog post. So for example this particular blog post’ URL is

https://kudithipudi.org/2017/02/24/how-to-configure…press-permalinks/

instead of

https://kudithipudi.org/?p=1762

This works by default in Apache because WordPress puts in the required rewrite rules.

To get it work in Nginx, you have to add the following config in the Nginx site configuration

Under the / location context, add the following

try_files $uri $uri/ /index.php?$args;

This is essentially telling Nginx to try to display the URI as is, and if it fails that, pass the URI as an argument to index.php.

February 24, 2017 by HOWTO Linux Web 0

HOW TO : Query varnishlogs for requests with 404 responses

varnishlog, one of the tools provided with varnish cache, uses VSL Query Expressions (https://www.varnish-cache.org/docs/trunk/reference/vsl-query.html) to provide some powerful insights into the requests and responses.

Here is a how you can use varnishlog to show all client requests that are ending up with a 404 response.

sudo varnishlog -g request -i ReqURL -q "BerespStatus != 200"

Technically, this particular query shows all client requests with a response other than 200.

Breaking down the commands

-g request : shows all entries related to the request

-i ReqURL : forces varnishlog to only display the Requesting URL

-q “BerespStatus != 200” : query filter to only match non 200 responses. Note that the query has to be enclosed in “”.

February 1, 2016 by HOWTO Linux Networking Technology Uncategorized Web 0

HOW TO : Restrict access to proxied content in Apache

If you are using the mod_proxy feature in Apache to forward requests for certain content to a backend server, but want to restrict access to that content to clients originating from certain IP addresses, you can use the location feature in Apache.

The Location directive limits the scope of the enclosed directives by URL. This is very similar to the Directory directive, but the difference is that you can put controls based on the URL rather than the location of the content.

In this example, I am forwarding content destined to https://kudithipudi.org/testLocation to an internal server at http://127.0.0.1:8080/testLocation. I am going to use the Location directive to restrict access to just requests originating from IP Address 10.10.10.10

[code]

<Location /testLocation>
Order Deny,Allow
Deny from all
Allow from 10.10.10.10
</Location>

ProxyPass /testLocation http://127.0.0.1:8080/testLocation
ProxyPassReverse /testLocation http://127.0.0.1:8080/testLocation [/code]

 

January 29, 2013 by HOWTO Networking Technology Uncategorized Web 0