HOW TO : Query varnishlogs to show backend server

Command parameters for varnishlog to view the backend server that is processing the request. In this particular case, I wanted to see the request URL and backend server for any responses with HTTP code 401 (unauthorized access)

sudo varnishlog -i BackendOpen,BereqURL -q "BerespStatus == 401"

HOW TO : Search which package contains a filename

If you are using a Linux system that uses yum for package management (like Fedora, Centos, RHEL), you can use the following command to find out which package contains a file. This is useful when you want to figure out which package to install. For example, dig (DNS utility) doesn’t come pre-installed on the system. And running “sudo yum install dig” doesn’t do anything.

sudo yum whatprovides '*/dig'

This returns

Loaded plugins: fastestmirror
 Loading mirror speeds from cached hostfile
 32:bind-utils-9.8.2-0.47.rc1.el6.x86_64 : Utilities for querying DNS name servers
 Repo : base
 Matched from:
 Filename : /usr/bin/dig

breaking down the command options

whatprovides : Is used to find out which package provides some feature or file. Just use a specific name or a file-glob-syntax wildcards to list the packages available or installed that provide that feature or file.

HOW TO : Enable wildcard domains in Squid

We were trying to modify some ACL (access control lists) in squid to allow traffic to certain websites. Instead of adding each individual hostnames in a domain, we wanted to add all traffic to a certain domain.

Document on the interwebs is old or not clear on how to achieve this.

After some trial and error, here is what works

say you want to allow all traffic to the google.com domain, you create a access list using dstdomain like below

acl name_of_acl dstdomain .google.com

The “.” before the domain name acts as a wildcard

Then you use the acl to allow http access to it like below

http_access allow name_of_acl

HOW TO : Use awk to print values larger than certain number

Quick how to on using awk to filter results if a certain value (column) is larger than a set value.

For example, if you have a file (servers.txt) with lines in this format

a_datacenter, servers 20
 error, servers xyz
 b_datacenter, servers 21
 c_datacenter, servers 50

and you want to show only the lines that have server value larger than 20, you can do this in awk by running

grep datacenter servers.txt | awk '$3 > 20  {print ;}' | more

breaking down the commands

grep – parsing down the output to just show the lines containing datacenter

awk – $3 > 20 : Get the third variable (awk seperates text using spaces by default) and check if it is greater than 20

print – print the entire line

HOW TO : Find files, search for content in them, replace the content

The title pretty much says it all :). Here is a quick  one liner, using multiple tools, to look for files in a directory, search for certain content in them and replace them with other content

[code]find -type f | xargs grep -l ORIGINAL_CONTENT | xargs perl -p -i -e ‘s/ORIGINAL_CONTENT/NEW_CONTENT/g’ [/code]

You can theoretically take out the grep (second command) and directly pipe the find output to perl and get the same outcome.

Going over list of the options used

find

  • “-type f” lists all objects of type file in the directory (and sub directories)

grep

  • “-l” lists the names of the files (with relative path) which have the text ORIGINAL_CONTENT in them

perl

  • “-p” forces perl to loop through requests. In this case files
  • “-e” tells perl that the next argument is a perl statement
  • “-i” tells perls to edit the file in place (i.e. no need for an output file)