8.09: How can I hack libc.a to alter how hostnames are resolved?

[ formerly in section 1.618 ]

[ Editor's note: You might want to see Question 2.07 for advice on
recovering from a deleted or corrupted libc.a before attempting this
hack.  Note that this procedure is for AIX 3.2 ONLY --- AIX 4.x already
has a supported method of controling name resolution.  See question 1.800.]

1. get the resolv+ source (I see a copy on ftp.uu.net in networking/ip/dns
   and there are likely copies elsewhere).  We are using version 2.1.1,
   which appears to be the latest available.  gethostnamadr.c needs a couple
   of additions:

	23a24,26
	> #ifdef _AIX
	> #include <sys/time.h>
	> #endif
	35a39,41
	> #ifdef _AIX
	> #include <sys/ioctl.h>         /* for SIOCGIFCONF */
	> #else
	36a43
	> #endif

2. Use the following instead of the supplied shlib/Makefile:

LIBP=	gethostnamadr.o herror.o res_data.o res_query.o res_mkquery.o \
	sethostent.o res_send.o res_debug.o res_comp.o res_init.o
CFLAGS=	-O -D_BSD=43 -D_NO_PROTO -DNIS -DDEBUG -U__STR__

all: shr.o 

shr.o:	$(LIBP) setup 
	ld -o $@ /lib/syscalls.exp $(LIBP) tmp.o -bM:SRE -bE:shr.exp -bE:/lib/syscalls.exp -bI:crypt.imp -H512 -T512 -bh:4 -lc

setup:
	rm -f libc.a crypt.imp
	cp /lib/libc.a .
	chmod 755 libc.a
	ar xv libc.a shr.o
	/bin/dump -nv shr.o | grep EXP | awk '{print $$NF}' > shr.exp
	ld -o tmp.o -bnso shr.o -r
	@ echo '#!' > crypt.imp
	@ echo __crypt >> crypt.imp
	@ echo __setkey >> crypt.imp
	@ echo __encrypt >> crypt.imp
	
clean:
	rm -f shr.o tmp.o crypt.imp shr.exp $(LIBP) libc.a

install_libc:	install_libc.c
	cc -o $@ install_libc.c -bnso -bI:/lib/syscalls.exp

herror.o: ../herror.c
	$(CC) $(CFLAGS) -c ../herror.c

res_comp.o: ../res_comp.c
	$(CC) $(CFLAGS) -c ../res_comp.c

res_debug.o: ../res_debug.c
	$(CC) $(CFLAGS) -c ../res_debug.c

res_data.o: ../res_data.c
	$(CC) $(CFLAGS) -c ../res_data.c

res_init.o: ../res_init.c
	$(CC) $(CFLAGS) -c ../res_init.c

res_mkquery.o: ../res_mkquery.c
	$(CC) $(CFLAGS) -c ../res_mkquery.c

res_query.o: ../res_query.c
	$(CC) $(CFLAGS) -c ../res_query.c

res_send.o: ../res_send.c
	$(CC) $(CFLAGS) -c ../res_send.c

gethostnamadr.o: ../gethostnamadr.c
	$(CC) $(CFLAGS) -c ../gethostnamadr.c

sethostent.o: ../sethostent.c
	$(CC) $(CFLAGS) -c ../sethostent.c

strpbrk.o: ../strpbrk.c
	$(CC) $(CFLAGS) -c ../strpbrk.c

strerror.o: ../strerror.c
	$(CC) $(CFLAGS) -c ../strerror.c

3. As shipped, IBM's /lib/syscalls.exp contains an entry for fork().  This
   needs to be removed as it will cause the new shr.o to use the system call
   entry point rather than the library wrapper and this can cause some rather
   odd behavior.  For example, I ran across one using the '!' command in vi
   where the error/informational messages were corrupted.

4. You can use "ar r libc.a shr.o" but that will leave a big hole in libc.a,
   since the new shr.o is slightly bigger than the original.  I always extract
   all the .o's from libc.a and build a brand new one - suit yourself.

5. Before the next step, you'll want to set up the two configuration files.
   The first is /etc/resolv.conf and it is basically the same as before
   except for the new keyword "search" - intended to replace the "domain".  See
   the resolver.5 manual page for details (included with the resolv+ source).
   The other file is /etc/host.conf, which is where you set the order of
   search.  See resolv+.8 for information on this.

6. Now, the only tricky part left is to get the new libc.a installed.  You'll
   note the Makefile has a target for install_libc.  Just put the following
   in install_lib.c and run "make install_libc" to build.

------------------------------- begin install_libc.c -------------------------
#include <stdio.h>

static char *nodns[] = { "/usr/ccs/lib/libc.a" , "/usr/ccs/lib/libc.a.ORIG" };
static char *hasdns[] = { "/usr/ccs/lib/libc.a.NEW" , "/usr/ccs/lib/libc.a" };

#define OLD     (0)
#define NEW     (1)

main()
{

	if(link(nodns[OLD],nodns[NEW])) {
		perror("link");
		exit(1);
	}

	if(unlink(nodns[OLD])) {
		perror("unlink");
		exit(1);
	}

	if(link(hasdns[OLD],hasdns[NEW])) {
		perror("link");
		exit(1);
	}

	if(unlink(hasdns[OLD])) {
		perror("unlink");
		exit(1);
	}

	exit(0);
}
------------------------------- end install_libc.c ---------------------------

7. You'll want to do this on a quiet machine.  Move the new libc.a to
   /usr/ccs/lib/libc.a.NEW, then run install_libc.  It is probably a good
   idea to reboot afterwords (though not strictly required).  We have a similar
   program called restore_libc (exercise for the reader) for backing out the
   change.

   Rather than do this on every machine (we have 170+ RS/6000's), I simply
   put this new libc.a into my network-installable image, which I blasted
   out over the year-end holiday break (I created a method for loading a new
   image onto machines without having to boot off of floppies and turn keys).

   Of course, for the really cautious, you can always make the change after
   booting off of the maintenance floppies...