Web

Overheard : Data Silos and Personal Agents

I asked my Hermes agent to browse a Reddit thread on foods that feel healthy but aren’t, and turn the replies into a table. It worked great. No scrolling, no copy pasting, just a clean summary in seconds.

Small win, but it got me thinking.

I miss the old open internet. Information was easy to find, easy to link to, easy to share. That’s changing fast. Between AI chatbots (Gemini, ChatGPT, Claude) and walled networks like Discord and Reddit, more of the information people write for each other is ending up locked inside private data stores. You usually need to be a member of that network just to see it.

These platforms didn’t start out this way. They began as places to bring people together. Now that same user generated content is the thing being monetized, gated, and repackaged, often with little coming back to the people who actually wrote it.

This is where personal agents can help. Something like Hermes doesn’t just automate one task. It can move across silos on your behalf: checking Reddit, Discord, a chatbot’s closed ecosystem, wherever the information lives, and bringing it back to you in one place, in a format you can use.

It doesn’t fix the siloing. The incentives behind it aren’t going anywhere. But it gives us a way to navigate the fragmentation without joining every network or trusting every walled garden. Personal agents may be the closest thing we get back to the open internet, not because the data opens back up, but because something is finally doing the work of stitching it together for us.

Food that feels healthy but isn’t

#FoodMentionsWhy It’s Bad
1Yogurt29Flavored yogurts are loaded with added sugar — some more than a candy bar. “Fruit on the bottom” is jam. Low-fat versions often swap fat for sugar.
2Granola25Marketed as wholesome, but packed with sugar, oil, and calories — a small bowl can hit 400+ calories.
3Salad20The greens are fine, but dressing, croutons, cheese, fried chicken, and bacon bits can push restaurant salads past 1,000 calories.
4Smoothie14Store-bought versions can have 50-80g of sugar — more than a soda. Blending destroys fiber, so sugar hits the bloodstream fast. Homemade is generally fine.
5Veggie Chips/Straws10Made from potato starch and vegetable powder — nutritionally close to potato chips, just dyed and marketed as vegetables.
6Fruit Juice10Stripped of fiber, leaving concentrated sugar. Orange juice has ~22g sugar per cup — even 100% juice.
7Sports Drinks (Gatorade)8Designed for athletes burning 1,000+ calories. For sedentary people, it’s mostly sugar water with food coloring.
8Honey / Agave7“Natural” doesn’t mean healthy — the body processes them almost like table sugar. Agave is especially high in fructose.
9Dried Fruit7Removing water concentrates the sugar. A handful of dried mango can equal the sugar in several whole mangoes.
10Cereal7Even “healthy” cereals (Special K, Raisin Bran, Cheerios) are ultra-processed with added sugar and stripped nutrients.
11Trail Mix4Designed as high-calorie fuel for long hikes, not desk snacking. Candy pieces and sugar-coated nuts make it candy in disguise.
12Sushi4White rice, sugary rice vinegar, high-sodium soy sauce, and spicy mayo add up — a spicy tuna roll can be 500+ calories.
13Peanut Butter4“Natural” labels can be deceptive — some commercial brands replace healthy oils with hydrogenated oils and sugar.
14Margarine4Made from industrially processed vegetable oils, historically high in trans fats.
15Diet Soda3Zero calories, but artificial sweeteners may disrupt gut bacteria and trigger cravings, potentially leading to overeating later.
16Coconut Oil3About 90% saturated fat — higher than butter — despite its “superfood” reputation.
17Avocado Toast3Café versions can hit 800+ calories with thick bread, excess avocado, and added oils/toppings.
18Frozen Yogurt2Marketed as a healthy ice cream swap, but toppings like cookie dough and syrup turn it into a sugar bomb.
19Muffins2Essentially cake without frosting — a Starbucks blueberry muffin can have 350-500 calories and 30g+ sugar.
20Protein Bars2Many are candy bars with protein powder added — check for sugar near the top of the ingredient list.

HOW TO : Sort Amazon search results by number of customer reviews

When browsing products on Amazon, sorting by average rating doesn’t always surface the most popular or widely purchased items. A simple URL trick lets you sort results by the total number of customer reviews, which is often a better indicator of popularity.

Add “&sort=review-count-rank” at the end of the URL and it will sort the results by the total number of customer reviews (and not the averating rating). Helpful to identify popular products.

Only works when sorting results from a department (ex: Tools & Home Improvement›Safety & Security›Flashlights›Handheld Flashlights)

Collection of tools to serve local content on your workstation to the Internet

A quick collection of tools you can use to serve/publish content/applications on your local dev to the Interwebs. Some use cases for these types of tools..

  • Developed a static website and want to show it someone that is not right next to you.
  • Developed an API that you want an app or user to access from the web

List of tools:

  • https://ngrok.com/ : Most popular tool for this purpose. the free tier is enough for most use cases.
  • https://tunnelto.dev/ : Latest entrant in this space. In addition to have a paid hosted service, you can run this for free on your own server. But that defeats the pupose of having a tool to use in a pinch to share content :).
  • http://pagekite.net/ : Been around for 10+ years. Similar to tunnelto.dev, you can run this on you own server or pay (very nominal price) for the hosted service.

HOW TO : Configure nginx to use URI for modifying response content

That was a pretty long title for the post :). I love nginx for it’s flexibility and ease of use. It is like a swiss army knife.. can do a lot of things :).

We needed to serve some dynamic content for one of our use cases. If user visits a site using the following URL format http://example.com/23456789/678543 , we want to respond with some html content that is customized using the 23456789 and 678543 strings.

A picture might help here

Here’s how this was achieved

  • Define a location section in the nginx config to respond to the URL path specified and direct it to substitute content
    location ~ "^/(?<param1>[0-9]{8})/(?<param2>[0-9]{6})" {

            root /var/www/html/test/;
            index template.html;
            sub_filter_once off;
            sub_filter '_first_param_' '$param1';
            sub_filter '_second_param_' '$param2';
            rewrite ^.*$ /template.html break;
    }

create a file named template.html with the following content in /var/www/html/test

Breaking down the config one line at a time

location ~ "^/(?<param1>[0-9]{8})/(?<param2>[0-9]{6})" : The regex is essentially matching for the first set of digits after the / and adding that as the value for variable $param1. The first match is a series of 8 digits with each digit in the range 0-9. The second match is for a series of 6 digits with each digit in the range 0-9 and it will be added as the value for variable $param2

root /var/www/html/test/; : Specifying the root location for the location.

index template.html; : Specifying the home page for the location.

sub_filter_once off; : Specify to the sub_filter module to not stop after the first match for replacing response content. By default it processes the first match and stops.

sub_filter 'first_param' '$param1'; : Direct the sub_filter module to replace any text matching first_param in the response html with value in variable $param1.

sub_filter 'second_param' '$param2'; : Direct the sub_filter module to replace any text matching second_param in the response html with value in variable $param1.

rewrite ^.*$ /template.html break; : Specify nginx to server template.html regardless of the URI specified.

Big thanks to Igor for help with the configs!!

Why ADP?

ADP is a $70B+ (by market cap as of August 2019) company and yet cannot get a simple redirect correct. If someone that is asked to use it’s employee performance management system types in tms.adp.com (like most people would do), they get this nice friendly error

If by some magical and mystical reason, they type in https://tms.adp.com, they get this login page

I find it mind boggling that such a mature company cannot figure out

  1. Customer experience
  2. 301/302 http redirects
  3. HTTP Strict Transport Security (HSTS)

End Rant and sorry to all my friends that work at ADP 🙂

Optimizing cache infrastructure

I love when engineering teams share their tricks of trade for other organizations to benefit. While this might seem counter-intuitive, sharing knowledge makes the entire ecosystem better.

Etsy‘ engineering team does a great job of publishing their architecture, methodologies and code at https://codeascraft.com.

This particular article on how they optimize their caching infrastructure (https://codeascraft.com/2017/11/30/how-etsy-caches/) is pretty enlightening. I always thought the best method to load balance objects (app hits, cache requests, queues etc) to hosts was to use mod operations. In this blog post Etsy’ team talk about using consistent hashing instead of modulo hashing.

At a high level, it allows cache nodes to fail and not impact the overall performance of the application drastically in addition to making it easy to scale the number of nodes. This method is useful when you have a large amount of cache nodes.

More reference links

  • http://www.tom-e-white.com/2007/11/consistent-hashing.html
  • https://www.toptal.com/big-data/consistent-hashing
  • https://en.wikipedia.org/wiki/Consistent_hashing

 

HOW TO : Configure nginx for WordPress permalinks

Over the last week, I moved this blog from a LAMP (Linux, Apache, MySQL, PHP) stack to LEMP (Linux, Nginx, MySQL, PHP) stack. Have a blog post in the works with all the gory details, but wanted to quick document a quirk in the WordPress + Nginx combination that broke permalinks on this site.

Permalinks are user friendly permanent static URLs for a blog post. So for example this particular blog post’ URL is

https://kudithipudi.org/2017/02/24/how-to-configure…press-permalinks/

instead of

https://kudithipudi.org/?p=1762

This works by default in Apache because WordPress puts in the required rewrite rules.

To get it work in Nginx, you have to add the following config in the Nginx site configuration

Under the / location context, add the following

try_files $uri $uri/ /index.php?$args;

This is essentially telling Nginx to try to display the URI as is, and if it fails that, pass the URI as an argument to index.php.