Starting with a computer that has plenty of disk space for this role will obviously be the most beneficial.  In my lab, that was a Dell PowerEdgeM600 with a couple of 146G drives installed – configured as Raid0.  This blade server is installed inside a M1000 chassis.  After downloading the NAS4Free iso, I proceeded to launching the iDRAC GUI for the server.  For anyone not familiar, this is a remote console that allows you to do just about everything that you could do if sitting in front of the machine.

After the console was open and connected, I launched the virtual media wizard and connected the cdrom to the NAS4Free iso.  This step is just like inserting a CD into the local drive.  After the virtual CD was connected, the server was restarted and booted from it.

NAS4Free will present a menu after loading the FreeBSD kernel and associated modules. I selected option 1 for booting via normal mode .  After booting, another menu is presented that is titled Console setup.  The first step we want to do is option 9 – Install/Upgrade NAS4Free from LiveCD.  Another menu called NAS4Free Install & Ugrade Options is presented, in which we want to select option 2 – Install ‘Embedded’ OS on HDD/Flash/USB + DATA + SWAP partition.

By selecting this option, the installer will perform the following:

- Create MBR partition 1, using UFS, 120MB size for OS image
- Create MBR partition 2, using UFS, for DATA
- Create MBR partition 3, as SWAP
- Creates/Uses a RAM disk to limit read/write access to the device

This option will erase ALL partitions and data on the destination disk.

Note, I used option 2 because there wasn’t any other media installed in this server to put the OS on.  Therefore, the internal disks are being used for both the OS and the data storage, separated on different partitions.  In a optimal scenario, you could select the option to install onto a USB key – so that you could boot from that and use the internal storage solely for data storage.

Select CD/DVD drive for installation

Select media where NAS4Free OS should be installed (da0)

Do you want to add a swap partition – answer No

After the installation is completed, there are instructions on how to mount the disk/partition that was created for the data store.  Exit the Menu and select to reboot the system via the console setup menu.

All configuration is performed via a web interface after an initial network adapter is configured.  When the machine has completed booting from disk, select option 1 from the Console setup menu to Configure Network Interfaces.  Select the adapter that will be used as the initial/primary network interface.  If there are additional network interfaces connected and available, you may select them next (OPT1, etc.) – otherwise at the Configure OPT interface, select none and OK and then OK once more.

Next, select Configure Network IP addresses from the Console setup menu.  Answer the questions according to the characteristics of your individual network.  After answering all questions, the interface will initialize and you will be presented with a notice containing the URL for accessing the administration portal.

Access the URL from a web browser and login with the default credentials: admin/nas4free

System –> General

Set Hostname/Domain, and Time/Timezone

Disks –> Management

Click Plus sign to add disk

In the Disk field, select the disk da0 from the drop down menu.  Leave everything else as the defaults (including the Preformatted file system field as Unformated).  Click on the Add button, then apply changes.

Disks –> Mount Point

Click Plus sign to add Mount Point

In the Disk field, select da0 from the dropdown menu.

In the Partition type field, select the MBR partition option.

In the Partition number field, enter the number 2

In the Mount point name, you may enter whatever you wish here – which will be used as the subdirectory under /mnt to access this partition.  In this step, I entered: data (so the mount point will ultimately be /mnt/data/).  Click on the Add button, then apply changes.

Services –> iSCSI Target

Check the Enable box in the iSCSI Target header (do NOT check the Enable box in the iSCSI Target Logical Unit Controller header toward the bottom of the page).

Base Name –  I left this as the default: iqn.2007-09.jp.ne.peach.istgt

Click on the Save and Restart button at the bottom of the page.

Services –> iSCSI Target –> Targets

Extent, click on plus sign

Extent Name – change to whatever you wish, I left as the default (extent0)

Type: File

Path: /mnt/data/extent0 (this path may be different, according to what you named the mount point in a previous step)

File Size: Enter a size for the file, which will be created on the filesystem to store all the data in.  The size you enter here will be what is available to the iSCSI initiators.  Obviously, the size needs to be a value equal to or less than the available space on your mount point.  I had 280G free on /mnt/data and decided to make the file size 200G, so that I could come back and make another 50+ gig lun at another time.  Click on Add and then Apply changes

Services –> iSCSI Target | Portal Group | Add

Configure as necessary or Leave Defaults if OK and click Add. Explanation: this is basically asking you where the initiator will run, so by default it will be the interface IP:3260 .  Click on Add and apply changes

Services –> iSCSI Target –> Initiators

Click on the plus sign under Initiator Group

Leave defaults  Explanation: This is where you would configure allowed source addresses/networks that initiators are allowed to connect from.

Click Add

Services –> iSCSI Target –> Target

Click on the Plus sign under Target

Configure as necessary or Leave defaults and click on Add and Apply changes

You should now be able to successfully connect from an iSCSI initiator to this disk!

To do this in vmware, configure an iSCSI Software adapter on the host, configure the VMkernel Port Bindings in the Network Configuration tab.  Finally, on the Static Discovery tab – click on Add.  Enter the IP of the NAS4Free machine (otherwise known as the IP configured in the Portal Group previously).  For the iSCSI Target Name, you can find it in the web administration of the NAS4Free  server by navigating to:

Services –> iSCSI Target.  Under the Target section in the Name column will list the value that goes in the the iSCSI Target Name field.  In my scenario it was: iqn.2007-09.jp.ne.peach.istgt:disk0.  Click on Add and then OK to rescan. Once scan is complete and you see a connection, navigate over to the Storage Configuration in vCenter to Add the new Datastore.

The machine being built will be registered on RHN and entitled to download the updates we want to provide. Other RHEL machines on your local network can be updated from your local repository server without having to be registered on the RHN. In a nutshell, this machine will serve as a proxy to all other machines on your internal network for software updates.

Prerequisites:

Red Hat Linux OS installed on physical or virtual machine
At least 50G+ of available disk space (This will vary depending on how many repositories are kept)
Standard Server (minimal) installation
Apache installed and enabled

To get started installing the software (mrepo) that will download binaries from RedHat, install the following dependencies:

yum install pyOpenSSL
yum install rhn-client-tools

Download rhpl from: ftp://ftp.univie.ac.at/systems/linux/dag/redhat/el6/en/x86_64/dag/RPMS/rhpl-0.221-2.el6.rf.x86_64.rpm

rpm -i rhpl-0.221-2.el6.rf.x86_64.rpm
yum install createrepo

By default, mrepo will use /var/mrepo as the root for the software repository. I chose to add a separate disk just for this and mounted it at /var/mrepo before performing the mrepo installation

Download mrepo from:
ftp://ftp.pbone.net/mirror/dag.wieers.com/redhat/el6/en/x86_64/testing/RPMS/mrepo-0.8.8-0.pre1.el6.rft.noarch.rpm

Install mrepo rpm

rpm -i mrepo-0.8.8-0.pre1.el6.rft.noarch.rpm

Create directories:

[root@xlrepo01 mrepo]#mkdir /var/mrepo/rhel6s-i386
[root@xlrepo01 mrepo]#mkdir /var/mrepo/rhel6s-x86_64

System ID’s will be generated for each release that a mirror is kept for. This is essentially configuring an inventory object that will be used to ‘mock’ a real machine for entitlement purposes. Note, you still have to be entitled for every release that you want to mirror. This step only eliminates the need to have a source machine of each type that will be mirrored.

This first id will be for a 32bit RHEL 6 server

[root@xlrepo01 mrepo]# gensystemid -r 6Server -a i386 /var/mrepo/rhel6s-i386/
RHN Username: xxxxx
RHN Password: xxxxx
Writing out file /var/mrepo/rhel6s-i386/systemid

Next, we will generate an id for the 64bit RHEL 6 server

[root@xlrepo01 mrepo]# gensystemid -r 6Server -a x86_64 /var/mrepo/rhel6s-x86_64/
RHN Username: xxxxx
RHN Password: xxxxx
Writing out file /var/mrepo/rhel6s-x86_64/systemid

Copy the sample configuration file included to the defined mrepo configuration directory. This configuration file is specific to the distribution that will be downloaded and replicated. If you choose to replicate multiple versions of RHEL, then a separate configuration file is required for each. You will find many samples in the /usr/share/doc directory that can be copied into the correct location and used.

cp /usr/share/doc/mrepo-0.8.8/dists/rhel6s.conf /etc/mrepo.conf.d/

Now run mrepo for the first time to download the repositories:

[root@xlrepo01 data]# mrepo -ugv
rhel6s-i386: Updating Red Hat Enterprise Linux Server 6 (i386)
rhel6s-i386: Mirror packages from rhns:///rhel-i386-server-6 to /var/mrepo/rhel6s-i386/updates
rhel6s-x86_64: Updating Red Hat Enterprise Linux Server 6 (x86_64)
rhel6s-x86_64: Mirror packages from rhns:///rhel-x86_64-server-6 to /var/mrepo/rhel6s-x86_64/updates
rhel6s-i386: Generating Red Hat Enterprise Linux Server 6 (i386) meta-data
rhel6s-x86_64: Generating Red Hat Enterprise Linux Server 6 (x86_64) meta-data
[root@xlrepo01 data]#

Warning: This will take a while to run for the first time

You should now be able to browse to http://hostnameofreposerver/mrepo/

By clicking on one of the repository links, you should see something like the following:

Notice in the previous image:

At the top of the page contains the necessary information for a RHEL machine’s apt or yum configuration file, in order to install packages from this server.

Now let’s move to a client machine that will receive updates from the new update server that we just created. By default, there are (at least) a couple of files in the /etc/yum.repos.d/ directory.

[root@xlwww01 yum.repos.d]#ls -al
-rw-r--r--. 1 root root 22102 Aug  6 07:30 redhat.repo
-rw-r--r--. 1 root root   529 May 30 13:19 rhel-source.repo

Move these files out of the yum configuration directory:

[root@xlwww01 yum.repos.d]#mv r*.repo $HOME

Create a new repository configuration file that will point to the new local repository server:

[root@xlwww01 yum.repos.d]#vi xlrepo01.repo

This is what the configuration file would look like:

[rhel-6-server-rpms]
name = Red Hat Enterprise Linux 6 Server (RPMs)
baseurl = http://xlrepo01.xpresslearn.com/mrepo/rhel6s-x86_64/RPMS.all
enabled = 1

Now the update process can be run from the client machine:

[root@xlwww01 yum.repos.d]# yum update
Loaded plugins: product-id, refresh-packagekit, security, subscription-manager
Updating certificate-based repositories.

Please use yum-config-manager to configure which software
repositories are used with Red Hat Subscription Management.

Repository rhel-6-server-rpms is listed more than once in the configuration
rhel-6-server-cf-tools-1-rpms                   | 2.9 kB     00:00     
rhel-6-server-rhev-agent-rpms                   | 2.9 kB     00:00     
Setting up Update Process
Resolving Dependencies
--> Running transaction check
---> Package bind-libs.x86_64 32:9.8.2-0.10.rc1.el6_3.1 will be updated
---> Package bind-libs.x86_64 32:9.8.2-0.10.rc1.el6_3.2 will be an update
---> Package bind-utils.x86_64 32:9.8.2-0.10.rc1.el6_3.1 will be updated
---> Package bind-utils.x86_64 32:9.8.2-0.10.rc1.el6_3.2 will be an update
---> Package dhclient.x86_64 12:4.1.1-31.P1.el6 will be updated
---> Package dhclient.x86_64 12:4.1.1-31.P1.el6_3.1 will be an update
---> Package dhcp-common.x86_64 12:4.1.1-31.P1.el6 will be updated
---> Package dhcp-common.x86_64 12:4.1.1-31.P1.el6_3.1 will be an update
---> Package krb5-libs.x86_64 0:1.9-33.el6 will be updated
---> Package krb5-libs.x86_64 0:1.9-33.el6_3.2 will be an update
---> Package krb5-workstation.x86_64 0:1.9-33.el6 will be updated
---> Package krb5-workstation.x86_64 0:1.9-33.el6_3.2 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================================================
 Package                  Arch          Version                    Repository           Size
==============================================================================================
Updating:
 bind-libs                x86_64        32:9.8.2-0.10.rc1.el6_3.2  rhel-6-server-rpms   871 k
 bind-utils               x86_64        32:9.8.2-0.10.rc1.el6_3.2  rhel-6-server-rpms   181 k
 dhclient                 x86_64        12:4.1.1-31.P1.el6_3.1     rhel-6-server-rpms   317 k
 dhcp-common              x86_64        12:4.1.1-31.P1.el6_3.1     rhel-6-server-rpms   140 k
 krb5-libs                x86_64        1.9-33.el6_3.2             rhel-6-server-rpms   712 k
 krb5-workstation         x86_64        1.9-33.el6_3.2             rhel-6-server-rpms   414 k

Transaction Summary
==============================================================================================
Upgrade       6 Package(s)

Total download size: 2.6 M
Is this ok [y/N]: y

To recap, we can communicate with anything on the outside network – because the test machine is on the network with the inside interface that has a high security level (100) and we are going over a lesser trusted interface (outside/security level 0) for internet access. By default, the ASA has an implicit allow for this behavior, which is why we can communicate to the internet. Another default behavior, all traffic originally sourced (i.e. otherwise un-inspected) from a lower security level interface (outside) going to a higher security level interface (inside) will be blocked.

Let’s start by creating a basic policy that will deny all communications, apply it, and test for proper functionality.

logging buffered informational
logging on
!
access-list ACL_INSIDE_OUTBOUND remark - Block all and log
access-list ACL_INSIDE_OUTBOUND extended deny ip any any log
!
access-group ACL_INSIDE_OUTBOUND in interface inside

In the configuration above, we started by configuring the ASA to log everything with an assigned severity of informational (severity=6) and above to the internal buffer on the ASA. Before the logging will actually take place, it must also be turned on globally.

Next, we created an access list called ACL_INSIDE_OUTBOUND, the naming convention used was based on the following:
ACL(Type of object)_INSIDE(The interface which the ACL will be applied)_OUTBOUND(The direction of the traffic relative to the applied interface). It is a recommended practice to use all caps for the ACL name, since it is easier to spot and read. It is also good practice to label the ACL with a name that describes function and/or use.

The acutal ACL consists of two lines, the first is a comment. It is good practice to comment each entry in the ACL, so that you have a description of the intended purpose. The comment should come just prior to the access control entry, for easy reading.

The access control entry says to deny IP communications between any source/destination and serves as the last line in the access-list. All additional access control entries will go before this one and will typically be permit statements. This way, we will put a permit entry in for each allowed source/destination, so that any other undefined traffic gets ‘caught’ by the last access control entry and will be denied/logged.

If you come from a CheckPoint background, this rule would typically be called the ‘clean up’ rule. I can only assume this phrase comes from the thought that if we didn’t explicitly allow (or deny) specific traffic prior to the ‘clean up’ rule, then it will get cleaned up (denied) and discarded.

The last line applies the ACL inbound to the inside interface. The rule of thumb here is you want to decide as early as possible what to do with the traffic the ASA inspects. From the internal hosts, the first interface they will communicate with on the ASA is the inside interface. This is where we want to determine if the traffic is to go any further on the network.

With the ACL applied, we will attempt to access the Internet from our test machine on the inside network. After launching Internet Explorer and attempting to load www.google.com, we eventually get a message stating the page can’t be displayed. On the asa, we look at the log for possible entries:

stat-asa-01# show log
Syslog logging: enabled
    Facility: 20
    Timestamp logging: disabled
    Standby logging: disabled
    Debug-trace logging: disabled
    Console logging: disabled
    Monitor logging: disabled
    Buffer logging: level informational, 7 messages logged
    Trap logging: disabled
    Permit-hostdown logging: disabled
    History logging: disabled
    Device ID: disabled
    Mail logging: disabled
    ASDM logging: disabled
%ASA-5-111008: User 'enable_15' executed the 'logging on' command.
%ASA-5-111010: User 'enable_15', running 'CLI' from IP 0.0.0.0, executed 'logging on'
%ASA-6-106100: access-list ACL_INSIDE_OUTBOUND denied udp inside/10.1.100.100(56576) -> 
outside/8.8.8.8(53) hit-cnt 1 first hit [0xd00d8b73, 0x0]

The log buffer shows a single deny entry showing our workstation (10.1.100.100) trying to reach 8.8.8.8 using port 53. The first thing the workstation did is try to resolve www.google.com, which means it would need to communicate with the configured DNS server (in this case 8.8.8.8) to resolve the name. Since this traffic was blocked by the ASA, the workstation was unable to resolve this name to an IP address in order for the browser to attempt a connection.

Next, let’s allow our inside hosts access to the DNS server we have configured.

object network obj_host_dns_servers
 host 8.8.8.8 
object service obj_svc_dns
 service udp destination eq domain
!
access-list ACL_INSIDE_OUTBOUND line 1 remark - Allow DNS queries
access-list ACL_INSIDE_OUTBOUND line 2 extended permit object obj_svc_dns object obj_net_Inside object 
obj_host_dns_servers log 

In the configuration above, the first thing we did was to define some additional objects. In the previous article, we introduced this concept and built and object for the network assigned to the inside interface. Here we are going to add two additional objects:

– a ‘network’ object that defines a single host, which in this case, is our dns server
– a service object that defines the port used to communicate with the dns server

Once the objects are created, we move onto adding to the existing access list named ACL_INSIDE_OUTBOUND. Remember, we want the new entries to go above the previously created entries. By default, if no line number is specified the new line will go at the bottom of the access list – thus causing it not to work, because the deny would come before the newly created permit statement.

So if we do a show access list before adding the additional lines, we see the following:

my-asa-01(config)# show access-list
access-list cached ACL log flows: total 8, denied 8 (deny-flow-max 4096)
            alert-interval 300
access-list ACL_INSIDE_OUTBOUND; 1 elements; name hash: 0x156432a3
access-list ACL_INSIDE_OUTBOUND line 1 remark - Block all and log
access-list ACL_INSIDE_OUTBOUND line 2 extended deny ip any any log informational

In order to put the comment on the line before the current number one, we just specifiy the line number we want the entry to appear before. So, a new line 1 entry will be placed before the current line 1, thus making the original become line number 2.

If we looked at the access list immediately after adding the new comment, it would look like the following:

my-asa-01(config)# show access-list 
access-list cached ACL log flows: total 7, denied 7 (deny-flow-max 4096)
            alert-interval 300
access-list ACL_INSIDE_OUTBOUND; 1 elements; name hash: 0x156432a3
access-list ACL_INSIDE_OUTBOUND line 1 remark - Allow DNS queries
access-list ACL_INSIDE_OUTBOUND line 2 remark - Block all and log
access-list ACL_INSIDE_OUTBOUND line 3 extended deny ip any any log informational

Now the additional permit statement, which we want between the two comments numbered line 1 & 2, is numbered line 2 so that it will be placed before the current #2, which will become line #3

stat-asa-01(config)# show access-list 
access-list cached ACL log flows: total 9, denied 0 (deny-flow-max 4096)
            alert-interval 300
access-list ACL_INSIDE_OUTBOUND; 2 elements; name hash: 0x156432a3
access-list ACL_INSIDE_OUTBOUND line 1 remark - Allow DNS queries
access-list ACL_INSIDE_OUTBOUND line 2 extended permit object obj_svc_dns object obj_net_Inside 
object obj_host_dns_servers log
  access-list ACL_INSIDE_OUTBOUND line 2 extended permit udp 10.1.100.0 255.255.255.0 host 8.8.8.8 eq 
domain log
access-list ACL_INSIDE_OUTBOUND line 3 remark - Block all and log
access-list ACL_INSIDE_OUTBOUND line 4 extended deny ip any any log informational

Notice something else we see while looking at the access list after the last permit entry (line 2) that was made. In line two, we see the permit statement as entered, now the very next line is indented and shows up as line 2 as well. This line basically prevents us from having to go back and look up all the objects to tell what is actually being referenced, it does this for us! So, I can look at this and see the source IP, destination IP, and port number. This provides a very handy and quick reference, I’ll say it is Cisco’s way of making up for the fact of pushing the whole object game on us in the first place

Now that the new access list is in place, we should be able to get further down the path of allowing access. From the workstation, another attempt to reach www.google.com still doesn’t work, as expected. However, the log should reflect progress:

my-asa-01# show log
%ASA-6-106100: access-list ACL_INSIDE_OUTBOUND permitted udp inside/10.1.100.100(51761) -> 
outside/8.8.8.8(53) hit-cnt 1 first hit [0x895fa5e7, 0x0]
%ASA-6-302015: Built outbound UDP connection 3808 for outside:8.8.8.8/53 (8.8.8.8/53) to 
inside:10.1.100.100/51761 (122.163.21.121/51761)
%ASA-6-302016: Teardown UDP connection 3808 for outside:8.8.8.8/53 to inside:10.1.100.100/51761 duration 
0:00:00 bytes 160
%ASA-6-106100: access-list ACL_INSIDE_OUTBOUND denied tcp inside/10.1.100.100(2088) -> 
outside/74.125.65.99(80) hit-cnt 1 first hit [0xd00d8b73, 0x]

You can see from the log output that dns traffic is now allowed by the access list. The very next entry reflects the information for the connection built by the ASA between the two devices, followed by a tear-down message for the same connection. Question: How can we tell by looking at this log that our newly created dns allow entry was successful (besides just going by the permit and connection built logs)? Notice the fourth and final entry in the log, a request was denied from our workstation to the IP address of 74.125.65.99 (Google) with a destination port of 80 (http). This means the workstation was successfully able to contact the DNS server (8.8.8.8) and resolve www.google.com (74.125.65.99), so the browser proceeds in making a request over port 80 to Google’s server. The http traffic was denied, since we haven’t added a permit statement for it yet.

Here is what the complete final configuration looks like for everything done thus far (both articles) to provide working Internet access with a base access list that can be built upon to form a complete access policy.

hostname my-asa-01
!
interface Ethernet0/0
 switchport access vlan 100
!
interface Ethernet0/1
 switchport access vlan 5
!
interface Vlan5
 nameif inside
 security-level 100
 ip address 10.1.100.1 255.255.255.0 
!
interface Vlan100
 nameif outside
 security-level 0
 ip address 122.163.21.121 255.255.255.248 
!
object network obj_net_Inside
 subnet 10.1.100.0 255.255.255.0
object network obj_host_dns_servers
 host 8.8.8.8
object service obj_svc_dns
 service udp destination eq domain 
object service obj_svc_http
 service tcp destination eq www 
access-list ACL_INSIDE_OUTBOUND line 1 remark - Allow DNS queries
access-list ACL_INSIDE_OUTBOUND line 2 extended permit object obj_svc_dns object obj_net_Inside object
 obj_host_dns_servers log 
access-list ACL_INSIDE_OUTBOUND line 3 remark - Allow outbound HTTP traffic
access-list ACL_INSIDE_OUTBOUND line 4 extended permit object obj_svc_http object obj_net_Inside any log
access-list ACL_INSIDE_OUTBOUND line 5 remark - Allow ICMP
access-list ACL_INSIDE_OUTBOUND line 6 extended permit icmp object obj_net_Inside any log
access-list ACL_INSIDE_OUTBOUND line 7 remark - Block all and log
access-list ACL_INSIDE_OUTBOUND line 8 extended deny ip any any log 
logging enable
logging buffered informational
nat (inside,outside) source dynamic obj_net_Inside interface
access-group ACL_INSIDE_OUTBOUND in interface inside
route outside 0.0.0.0 0.0.0.0 122.163.21.126 1
!
policy-map global_policy
 class inspection_default
  ...
  inspect icmp

This article will start like most others I found and mentioned previously, with the intention of being current and somewhat future proof, since it is based on latest versions of software to date. However, as we all know, when the next major version appears – this article will also become dated, just as the others are. Maybe, just maybe, I can come back and revisit this article for a refresh from time to time – so that it can retain its usefulness.

If your ASA does not have the latest software, the first thing I would recommend is to go and get it. This is of course, assuming you have access to the Cisco software center via a valid SmartNet contract. If not, hopefully it shipped with something fairly current. Cisco will offer updates that anyone can obtain to fix a security flaw, so check to make sure it is at least the current revision.

The software version this device used to write the article is: ASA Version 8.4(3)

Let’s jump into the initial configuration:

Configure hostname of the device

hostname my-asa-01

Next, we will configure two of the available switchports on the 5505. The model I am using has a total of 8 switchports, with the last two ports able to provide Power Over Ethernet. The Layer3 configuration does not go on the port configuration, we will assign the switchports to a Vlan and then configure the vlan interface (SVC) seperately.

interface Ethernet0/0
switchport access vlan 100
!
interface Ethernet0/1
switchport access vlan 5

As you can see from the previous port configuration, we are using two vlans – 5 and 100. The vlan configuration comes next and is where we assign interface names, security levels, and IP information.

interface Vlan5
 nameif inside
 security-level 100
 ip address 10.1.100.1 255.255.255.0
!
interface Vlan100
 nameif Outside
 security-level 0
 ip address 122.163.21.121 255.255.255.248

Using the interface names ‘inside’ and ‘Outside’ tells the ASA the function of each interface. It knows that when an interface name is called ‘outside‘ that it is directly connected to the (untrusted) Internet. The same thing goes with the name ‘inside‘, the ASA knows this interface will be where all internal hosts will be and automatically makes it the most secure (using security-level command).

The only important thing to know about security levels is that by default, an interface can’t communicate with another interface if it has a higher security level. So, an interface with a security level of 0 (most un-trusted interface out there) can’t communicate with hosts behind an interface with a security level of 100 (most trusted interface out there). Now, in my opinion, the security-level thing is a little dated – but still very much in use. The reason I say this is because the security level doesn’t mean much if you have an explicit (user defined) firewall policy installed. More on that later…

The next thing we need is a NAT statement, because we are using private (RFC-1918) addresses on the internal network and want to communicate with other hosts on the Internet. To do this, we must translate a private, non-internet-routable address into a valid public address (which is assigned to the outside interface).

First the specific information, with the explanation to follow:

object network obj_net_Inside
 subnet 10.1.100.0 255.255.255.0
!
nat (inside,outside) source dynamic obj_net_Inside interface

The first thing in the block of configuration above (object definition) is not NAT specific, but is being used in the NAT configuration. I must say, even though the ‘object’ concept has been in the ASA software for some time now – it has probably been slow to adopt with people who stick to command line only configuration. This is kind of like telling a C programmer that it is time to switch to something more object oriented, like C++ – old habits die hard.

A nice thing with objects is it gives you a opportunity to name all the subnets defined in the ASA, where traditionally when using only access-lists, it did not provide the same capability. So, we have defined a network object named obj_net_Inside and told it the associated subnet is 10.189.87.0/24, which is the network associated with the inside interface.

Now, on to the actual NAT statement that says – perform a network address translation when traffic sources from the inside interface and is destined to the outside interface AND the source IP address matches the object called obj_net_Inside. When the match is made a dynamic translation is performed (PAT) using the IP address assigned to the outside interface of the ASA. This means any traffic from a machine on the internal network will look to a website on the Internet as if the source address is (in this case) 122.163.21.121. Multiple addresses on the inside network can all be translated to the same 122.163.21.121 IP address, because the source port for each request will be made unique (handled by the ASA). By having a unique source address for each request, the reply traffic from the Internet will have a destination port of the random source port used when the request originated. This port number will allow the ASA to lookup in the translation table which inside host the traffic belongs to.

Last, but not least, the ASA needs a default route for access to the Internet. The next hop defined in the route statement is the IP address of the ISP’s interface.

route outside 0.0.0.0 0.0.0.0 122.163.21.126 1

Let’s see what we have with all the previously mentioned bits combined

hostname my-asa-01
!
interface Ethernet0/0
switchport access vlan 100
!
interface Ethernet0/1
switchport access vlan 5
!
interface Vlan5
 nameif inside
 security-level 100
 ip address 10.1.100.1 255.255.255.0
!
interface Vlan100
 nameif Outside
 security-level 0
 ip address 122.163.21.121 255.255.255.248
!
object network obj_net_Inside
 subnet 10.1.100.0 255.255.255.0
!
nat (inside,outside) source dynamic obj_net_Inside interface
route outside 0.0.0.0 0.0.0.0 122.163.21.126 1

That is the basic configuration required to give a machine on the inside network internet access. However, it is far from complete. One of the very next things we need to do is define some actual firewall rules, that provides much finer control than just having security levels assigned to interfaces. By looking at the article size so far, I will leave that for part two of the series.

So, that’s about it for Part 1 – we should have subtitled it: Minimal ASA configuration for basic internet access.

With that being said, what do we need for internet access at this point? Let’s cover that real quick:

I will plug a laptop’s Ethernet interface into port 0/1 on the ASA. In the Ethernet adapter properties, we will assign a static IP, since so far we have nothing that will automatically assign it an IP address. My interface configuration is:

IP address: 10.1.100.100
Subnet mask: 255.255.255.0
Default Gateway: 10.1.100.1

For the DNS server, I will use Google’s public DNS, which is:

DNS1: 8.8.8.8

After configuring my laptop and then opening a command prompt, the first thing I try is to ping the default gateway:

#ping 10.1.100.1
!!!!! 5/5 successful
#

Next thing is let’s make sure I’ll be able to communicate with something on the internet. How about we try and ping the DNS server from the laptop (warning home made ping prompt):

#ping 8.8.8.8
..... 0/5 successful
#

Why doesn’t this work? I can ping 8.8.8.8 successfully from the console of the ASA, assuming all my configuration is correct, why can’t I ping from the laptop? Well, let’s try accessing the internet anyway: Opening a browser and typing in www.cisco.com displays Cisco’s website – success!!

How was my laptop able to resolve www.cisco.com if it couldn’t ping the address? Well, that was because the DNS traffic was ‘inspected’ by the ASA and was allowed in both directions. With the ping (ICMP) traffic, the request is allowed outbound to the internet, but the reply is denied by default, thus causing the ping timeout condition.

One way to fix is to add ICMP to the default ‘global inspection policy’. The protocols defined in this policy allows the ASA to ‘inspect’ all traffic leaving and will ‘keep track’ of it – so when the ICMP reply traffic comes back to the ASA, it will be (dynamically) allowed back in. The default policy does not inspect ICMP, so we will add it:

policy-map global_policy
 class inspection_default
  inspect icmp
  ...
  ...

Now, the ASA will know we initiated the ping from inside the network and will dynamically allow the reply back in from the internet:

#ping 8.8.8.8
!!!!! 5/5 successful
#

Was ICMP/ping required for internet access? No. Is it nice to have available in most environments? Absolutely.

We will pick up in the second article discussing firewalling and creating rules to control communications, which will include more information regarding the security level command mentioned previously.

Continue to Part II

RSA Authentication Manager provides an authentication mechanism consisting of a “token” – either hardware (e.g. a Keyfob) or software (application that provides same functionally as a keyfob).  A hardware or software token is assigned to an individual, which generates an authentication code at fixed intervals (usually 60 seconds) using a built-in clock and the tokens factory-encoded random key (known as the “seed”). The seed is different for each token, and is loaded into the corresponding RSA SecurID server (RSA Authentication Manager, formerly ACE/Server) as the tokens are purchased.

In this example, I are using RSA Authentication Manager 6.1, which is running on a purpose built appliance that uses Windows 2003 Server with the RSA server software installed.  This particular solution includes Funk Software’s Steel Belted Radius, which provides a radius authentication mechanism into RSA.  At the time of this writing, this particular appliance and software version is approaching end of life and has since been replaced with Authentication Manager 7.1.  In the appliance version of 7.1 (known as Authentication Manger 3.0), the operating system has moved to Linux with Authentication Manager 7.1 loaded on top of it.  Version 7.1/3.0 also includes a radius server that can be used for radius clients needing to utilize two-factor authentication.

Now, onto the client portion of software used to interface with the RSA server.  As previously mentioned, there are two options, with the first being to use the RSA provided authentication agent for Unix/Linux.  The agent is actually a module that hooks into PAM, which is the central authentication standard used in most modern Unix/Linux systems today.  This option provides the maximimum functionality and interfaces directly with the RSA protocol (which means the RADIUS server is not required).

The second option is to load a RADIUS module into PAM (pam_radius_auth), which would then communicate to the RSA server via it’s built in RADIUS server.  Why would you want to use this option over the first option presented?  The RSA provided client is only supported on  a couple of Linux platforms, namely Red Hat and SuSE, which are both RPM based.  So if you are using any other Linux distribution (Debian based, etc.), there is not an RSA provided option with this client software.

Most Linux software repositories will contain a PAM radius module, which prevents having to download source code and compiling programs.  I’m specifically working on a Debian based system, which includes the module libpam_radius_auth in it’s repository.  The following contains instructions for configuring the system:

First, install the module from the distributions repository:

root@localhost:~# apt-get install libpam-radius-auth
Running /usr/bin/apt-get install libpam-radius-auth
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Suggested packages:
  radius-server
The following NEW packages will be installed:
  libpam-radius-auth
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 24.7kB of archives.
After this operation, 127kB of additional disk space will be used.
Get:1 http://ftp.us.debian.org lenny/main libpam-radius-auth 1.3.16-4.4 [24.7kB]
Fetched 24.7kB in 0s (58.4kB/s)      
Selecting previously deselected package libpam-radius-auth.
(Reading database ... 36010 files and directories currently installed.)
Unpacking libpam-radius-auth (from .../libpam-radius-auth_1.3.16-4.4_amd64.deb) ...
Setting up libpam-radius-auth (1.3.16-4.4) ...

Now that the module is installed, it’s time to edit configuration files:

root@localhost:~# vi /etc/pam_radius_auth.conf

My default configuration file had two invalid entries defined to show the format. One of two entries in my configuration file was for 127.0.0.1 with a comment below it that read “having localhost in your radius configuration is a Good Thing”. I don’t know what that is supposed to mean, but don’t leave it in the configuration as we have no Radius server running on the local machine.

After removing the existing (sample) entries and replacing with valid server entries, which contained server[:port], shared_secret, and timeout (separated by tab) – save and close the file. A particular detail to note is that initially I set the timeout value to the displayed default of 3 (seconds). However, I experienced authentication failures until I changed that value to 5 (seconds) – after noticing timeout messages in /var/log/auth.log.

Next, we need to modify a PAM configuration file in order to specify the use of the RADIUS module when authentication occurs. Note: There are several services that make use of the PAM system for authentication. Therefore, by doing what I am about to explain could cause negative impact on an application that uses PAM to authenticate users. Further research should be performed to determine which configuration file the following commands should be placed in, based on what you want to secure.

In this example, there are no applications running that need auth services provided by PAM (such as an FTP server, HTTP server, SAMBA, etc). The goal is to define the use of a global policy that uses the RADIUS module for central authentication. If you wanted to only secure a particular service (like ssh logins for administrator shell access via sshd), a different file other than the following can be modified so to not disrupt any other PAM using applications installed on the machine.

Edit the /etc/pam.d/common-auth configuration file:

root@localhost:/etc/pam.d# vi common-auth

First, find the following line in the configuration:

auth	required	pam_unix.so nullok_secure

Insert a new line BEFORE/ABOVE the previous line and paste the following line into the file.

auth	sufficient	pam_radius_auth.so

Save and exit the file. The previous addition to the common-auth file tells PAM to use the RADIUS module for authentication first (since it is listed first in the configuration). By specifying ‘sufficient’ in our entry, PAM determines that a successful auth using this module is satisfactory, therefore no other modules defined in the configuration file need to be processed. However, if there is a failure from this module (user didn’t exist on RADIUS server), then continue processing entries in this configuration file. Note: By configuring this way, any locally defined users on the system will still authenticate successfully. Therefore, it is advised to only have local ‘emergency accounts’ defined, in case the machine completely looses communications with all configured RADIUS servers – you would still be able to log in with a local user. If you adopt this policy, obviously the people who know the credentials to the locally defined account(s) should be minimal – in order to force the use of individual (RADIUS defined) accounts.

Next, edit the /etc/pam.d/common-account file

root@localhost:/etc/pam.d# vi common-account

Find the line:

account	required	pam_unix.so

Insert the following BEFORE/ABOVE the previous line:

account	required	pam_radius_auth.so

Save and exit the file. The previous addition to the common-account file is to tell PAM to use the RADIUS module for any authorization requirement (like permitting access to a service based on time of day, etc.), prior to checking the local database. By specifying ‘required’ in our entry, PAM determines that the success of the module is required for the module-type facility (in this case authorization is the module-type) to succeed.

Next edit the /etc/pam.d/common-session configuration file:

root@localhost:/etc/pam.d# vi common-session

Find the line:

session	required	pam_unix.so

Insert the following lines BEFORE/ABOVE the previous line:

session	required	pam_radius_auth.so
session	required	pam_mkhomedir.so	skel=/etc/skel/	umask=0022

Save and exit the file. The previous addition to the common-session file is to define tasks to be performed at the start and end of a user control of a service. By specifying ‘required’ in our entry, PAM determines that the success of the module is required for the module-type facility (in this case services is the module-type) to succeed.

As you can see, we added a second module (pam_mkhomedir.so) to the common-session configuration file. This is required in order to have a home directory available for a RADIUS authenticated user. The module will run after a successful authentication and create the user home directory in /etc/skel with the appropriate permissions.

That will complete the configuration setup. There is one final step left that has to be performed for every user that will login to the system via RADIUS. Because RADIUS doesn’t provide a directory service, we have to have UID and GID information pre-populated on our system. This is accomplished by creating the username and groupname on the local system, which will assign the necessary unique user ID and group ID values (numbers).

For example, a user that needs to authenticate via radius using a login id of johnh (that belongs to an associated group called johnh) needs to have the following performed on the local system:

useradd johnh

That will do it, the user will now have an entry that is created in /etc/passwd and /etc/group with an automatically created (unique) ID number.

Before I go any further, it must be stated that commercial packages exist that can do this type of analysis for you.  These software programs usually import CheckPoint logs into a larger data-source and then run various reports against it.  While those packages are extremely valuable to the firewall administrator, often times it is cost prohibitive to the company they work for.  It will be my attempt to share a Do It Yourself, bare bones, just get it done, alternative approach to buying these costly software packages.

As far as prerequisites, not much is needed: I’ll be using a Linux workstation for the utilities, such as cat,grep, and others.  The log data will be imported into a SQLite database for analysis.  Everything I have mentioned thus far is available on a Windows workstation, but will require a little bit of work to find/install it.  My point here is: If you have not taken the plunge to set up a Linux ‘utility’ workstation yet – now would be a great time to knock that out.  Anyway, I will show all my examples and reference the procedure as if it is being performed from a Linux machine.  However, I think it will easily be adaptable to the Windows only administrator.  If not, I will do my best to clarify points as questions are asked.

First, we must get the logs in the format we can work with.  This will require exporting the current CheckPoint log file type to a delimited, plain text type.  The utility required for this will be located in the Firewall1 program directory on the SmartCenter management station.

From a command line on the SmartCenter machine, we want to change to the firewall log directory:

cd \fw1_install_dir\RXX\fw1\log

where fw1_install_dir = SmartCenter installation directory and XX = the version of SmartCenter installed (i.e. R75).

running the ‘dir’ command in this directory will give you the name of the available logfiles for export.  The file names will follow the format of YYYY-MM-DD_HHMMSS_XX.log, select the file for export and run the following:

fwm logexport -n -p -m raw -i [YYYY-MM-DD_HHMMSS_XX.log] -o [YYYY-MM-DD_HHMMSS_XX.txt]

The switches are explained below:

Usage:
fwm logexport [-d delimiter |-s] [-i filename] [-o filename] [-f|-t] [-x start_p
os] [-y end_pos] [-z] [-n] [-p] [-a] [-u unification_scheme_file] [-m (initial|s
emi|raw)]
Where:
-d  - Set the output delimiter. Default is ';'.
-s  - Set the delimiter to be ASCII character #255.
-i  - Input log file name. Default is the active log file, fw.log.
-o  - Output file name. Default is printing to the screen.
-f  - Only in case of active log file - Upon reaching end of file, wait for new
records and export them as well.
-t  - Same as -f flag, only start at end of file.
-x  - Start exporting at the specified position.
-y  - End exporting at the specified position.
-z  - Continue exporting the next records, in case of an error. Default is to stop exporting.
-n  - No IP resolving. Default is to resolve all IPs.
-p  - No port resolving. Default is to resolve all ports.
-a  - Export account records only. Default is export all records.
-u  - Unification scheme file name. Default is log_unification_scheme.C.
-m  - Unification mode: initial-order, semi-unified, or raw. Default is 'initial'.

The switches used in the previous example should be self-explanatory after looking them up using the syntax help above.

Here is the command I ran in my environment

C:\Program Files\CheckPoint\R71\fw1\log>fwm logexport -n -p -m raw -i "2011-09-28_235900_98.log"
 -o "d:\2011-09-28_235900_98.txt"
Starting... There are 20492936 log records in the file
File logexport.ini was opened successfully
Processed 20492936 out of 20492936 records (99%)

Once I did this in my environment for one log file, which contained access information for a 24 hour period, the result was a 5.2G text file.  This would obviously be impossible to open with any editor, which is where our Unix utilities come into play.

At this point, I only want to load the necessary data into the db.  This keeps the database small and makes queries much more responsive.  In order to extract a subset of data from the log output, we will use awk and grep to put the desired results into a separate file.  In this example, I want traffic that was either sourced or destined to 10.16.2.20.

# awk '{q=split($0,a,";");if (NR==1){for (v=1;v<=q;v++) c[a[v]]=v} printf("%s;%s;%s;%s;%s;
%s;%s;%s;%s;%s\n",a[c["date"]],a[c["time"]],a[c["action"]],a[c["rule_uid"]],a[c["rule_name"]],
a[c["src"]],a[c["s_port"]],a[c["dst"]],a[c["service"]],a[c["xlatesrc"]])}' 2011-09-28_235900
_98.txt | grep '\<10.16.2.20\>' > windowsdc01.txt

The previous command uses ‘awk’ to process the file ’2011-09-28_235900_98.txt’ and only print the log fields we are interested in. Awk is being used because for some reason, Checkpoint does not export log files the same way twice.  For example a fwm export one day may contain 51 columns, the next day it might contain only 40.  Obviously this would play havoc on importing the same fields each time into our database.  By extracting just the columns we need, this ensures the same format each time.  This command looks very complex, the only thing you really want to consider is if additional fields are wanted in the output.  If this is the case, just make sure the additional fields are specified in order within the script.  For example, let say you want to add an additional field (i/f_name) to the output.  If you look at the first line of the original exported file, which are the column headers, you will see the “i/f_name” column is between “action” and “rule_uid”.  So hear is what you would add to the existing script (in bold)

a[c["action"]],a[c["i/f_name"]],a[c["rule_uid"]],

You will also need to add an additional %s; after the printf statement for each additional field you add

Moving on, note after the grep command the \< and the \> characters with the ip address in between.  What this does it tell grep to only match this character string if it’s the beginning or end of the word.  If the \< characters were missing from above, then we would also match other hosts like 110.16.2.20 or 210.16.2.20.  Likewise, if the \> were missing off the end, then we would match on 10.16.2.201, 10.16.2.202, and so on.  Finally, the greater than sign followed by a file name, will output the results to a file instead of to the default location of the screen.

Now, I have a separate file that contains only the data I care about at the moment and it is 28Mb vs. the 5Gb source file we started with.  The next thing to do is load it into a sqlite database.  Before we can do that, we have to create the database with a table containing the proper columns to accept the text file import.  We start by invoking sqlite and passing it a variable that will be the name of a new database, which in this example is called data.db.  Once sqlite is invoked, run the SQL script shown below, which is used to create the table. Obviously this sql statement would need to be modified if you added additional fields over what is shown in the previous example.

# sqlite3 data.db
SQLite version 3.7.5
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite>CREATE TABLE tbl_fwlogs(
f_date varchar(10),
f_time varchar(10),
f_action varchar(10),
f_rule_uid varchar(100),
f_rule_name varchar(15),
f_src varchar(10),
f_srcport varchar(10),
f_dst varchar(15),
f_service varchar(10),
f_xlatesrc varchar(10));
sqlite>

Define the separator used in the import file

sqlite> .separator ";"

Finally, import the text file into the database

sqlite> .import file.txt tbl_fwlogs

Now that you have data to query, here is a sample that displays what rules are being used in the rulebase for this particular host.

sqlite> select DISTINCT(f_rule_uid),f_rule_name from tbl_fwlogs;

{3C9A2260-8E75-4488-82C3-A3F279BB72B6};Srv to Srv access
{13385ECB-2S6F-4657-CC20-4DA76F217141};Windows Domain Resources
{3A5A0D9E-1D32-41BD-9795-829ED5CFE366};Time Requests
{5D2726D6-738A-43BA-8B5B-63FA0A7EBF78};Monitoring Servers
{A696790B-2605-46B2-BDA3-8A64A5B98C1A};DNS
{DDDCF882-8121-4E27-8A28-EA17EC5BC47E};Internal ICMP
sqlite>

In this example we see there are 6 rules in use for this host.  From here additional queries would determine src/dest addresses and protocols used so that we could take that info and build a stricter rule set for this host.

First, a few details regarding the example scenario:

A previous upgrade to an IP enabled KVM switch was causing issues with it’s normal operation. There were issues with local use using a directly attached keyboard/monitor/mouse and also when using the viewer plugin remotely. After my co-workers had complained enough, I decided it was time to downgrade the software to the previously running code, which did not have all the issues that was currently happening. Using the management software for the KVM, I downgraded 7 of 8 devices successfully. One device failed during the procedure and subsequently stopped responding on the network.

After giving sufficient time for possible self-recovery with no results, I decided it was time to investigate further. Upon inspecting the device visually, it was determined that the equipment was in recovery mode (The power light was blinking steady with no other lights on the device). This determination was made by going to the hardware manufacturers website and downloading the manual for my particular model, then looking up the device states in the troubleshooting section of the documentation.

The first thing attempted was an obvious one: Try and power cycle the hardware. After turning off and back on, the same result happened – a steadily flashing power light.

The documentation stated that when the device was in recovery mode, it would automatically attempt to download the system image via tftp from the management server. After inspecting the machine running the KVM management software, I was able to determine there was no traffic between it and the failed device. There are several ways to troubleshoot this, my particular method was to run a packet sniffer (Wireshark) from the management server to see if any requests were coming from the KVM’s IP address. If installing Wireshark (or similar program) is not an option on the machine, a portable version is available from the website that can be run out of a directory that either resides on a hard or flash drive.

At this point, a support call would have been the next course of action. However, a current maintenance contract did not exist on this equipment, so tech support was not an option. Truthfully, even if it was an option, I most likely wouldn’t be using it. I would rather be hung upside down (by my toenails), 30 feet in the air, with a pack of flesh eating Hyenas waiting underneath, for me to plummet to my death so they could consume me. Not that there is anything wrong with calling tech support, never mind – I digress…

The device is now officially ‘bricked’ (hence the title of this article). The urban dictionary defines the term as follows:

Bricked refers to ANY hardware that is unable to start up due to bad software; Usually because of a bad software flash, a modification done improperly, loss of necessary files, etc.

Thankfully, the majority of the time a device can be recovered after being in this state.

The next step in my process was to determine if a console was available. After looking at the documentation once again, I found that a serial port was available on this device for management purposes. After recording the applicable serial port settings and grabbing a null modem (serial) cable, it was off to the data center where the device was located.

My thought was to connect the serial cable between a laptop and the KVM device to see if I could get any output using a terminal program. Putty is my terminal program of choice, which has support for serial connections. I configured Putty to connect to COM1 at 9600 baud with 8 bits, No parity, and 1 stop bit (better known as eight, ‘n’, and one). The hope here was maybe the device used a bootloader which is a small piece of software that loads initially (like a BIOS) and in turn loads the full software image for the device. Many times when a bootloader can’t load the main software image, there is a very basic command line structure available to perform recovery functions such as transferring an image, re-issuing boot commands, etc.

After starting Putty and pressing the Enter key several times (which usually prompts the connected device to respond), there was no response. I’m still not sure what was going on with why the console wasn’t working, because I moved on from that very quickly. (My assumption here was the command line via serial port was only available after the firmware was correctly loaded and running on the device)

As I previously mentioned, by reading the documentation, I knew the device was supposed to request a boot image via TFTP. So, I took my laptop and connected it to an isolated switch along with the KVM device’s network interface. After starting Wireshark on the laptop and starting a capture, the KVM was powered on.

AH, progress!

The above image displays a WireShark window running on my laptop.  When this photo was taken, there was a display filter set – so that only traffic from the KVM src mac-address was shown.  (A mac filter was used, since that was the only known information).  The mac-address is always shown, usually via a sticker on the device.  Notice it has an IP of 10.0.0.2, which obviously is hard coded in the firmware – since I didn’t have a DHCP server running on the laptop. The next thing you see is the appliance making a request via TFTP to 10.0.0.3 (again another hard coded entry in the firmware) and is requesting a file with the name DSRxx20.fl.

With this information, the laptop’s network interface can now be set statically to 10.0.0.3. The next thing I needed was a TFTP server loaded on my laptop. This is an easy task, with several available freely on the Internet, download your favorite (my recommendation is tftpd32) TFTP server and run it.

The final step is to put the firmware for the device into the TFTP server ‘home’ directory and make sure the filename matches what is being requested (in this case it was DSRxx20.fl). After the file was in place with the TFTP server running, I power cycled the appliance once again:

As you can see from above, the transfer took place, which then the device proceeded to boot up perfectly! SUCCESS! Although this is not a universal step by step instruction on how to save any ‘bricked’ device – it should help outline the steps required to discover what is needed to bring something you are working on back to life.

First shut the running machine down, luckily my lab was running in a virtual environment and had the vmware tools installed on the guest machine that needed the password reset on.  I opened a console window to the vm and in the viewer selected VM from the menu bar, Power, then Restart Guest (Ctrl+R).  Obviously if this is a physical machine or a virtual without the tools installed, you may have to shut it down / power off not so gracefully.  However, at this point – if you can’t login to the machine – what else can you do?

Make sure your boot order is setup properly in the BIOS, so that the machine will attempt to boot from CD/DVD first, before the hard drive.  Once this is correctly set, be sure to press a key for booting to the DVD, while the message is shown telling you to do so.

Once booted to the DVD, the following screenshot will be the first thing you see.  Select the desired language and click on Next.

The following screen will appear next, which typically you would use the Install now on.  However, there are a couple more options on this screen.  In the lower left hand corner, click on the Repair your computer.

The following screen reflects the next window you will see, which is a question regarding the Windows installation to be repaired.  As you can see below, I have the Windows installation location selected (note it tells you what drive letter will be used during this repair session – D: in this/most cases).  Click on Next.

The following window will then appear, select the Command Prompt link.

After selecting a command prompt link, a cmd window will open like the one below:

Perform the following steps:

- Change to the assigned drive letter Windows is installed on.

- Change to the \Windows\System32 directory.

- Move the existing file utilman.exe to a temporary name, such as utilman.exe.bak

- Copy the command interpretor cmd.exe to utilman.exe

The previously described operations are reflected in the following screenshot:

Exit the command Window and click on the Restart button

After restarting, allow the machine to boot normally from the hard drive.  The following screenshot is the normal Windows login screen you will see.  Press the key combination Windows key + U

After pressing the Windows+U keys, a command window will appear like what is in the following screenshot:

The next step is your normal command line password change commands.  If your not aware of which username you should be changing, use net group “Domain Admins” to determine the userid that needs resetting.

The user Administrator is what I need reset, so use the following command net user Administrator <A secure password> like what is shown in the following screenshot:

At this point, you have successfully reset the Administrator password!  However, there is one more task that needs to be performed.  Shut the machine down and repeat the steps with the Windows 2008 Server install DVD up to the point where you have a command window opened.  Copy the backup file utilman.exe.bak over the existing utilman.exe file (which at this point is a renamed copy of cmd.exe).  Reboot the machine back to the OS installed on the hard drive.

Note:  Failure to perform this step will keep the what is now a potential security hole open on your machine.  So it is important to return the utilman.exe file back to it’s original state.

This file is already opened by (user name). Would you like to make a copy of this file for your use?

The reason I needed to close this file, was because I was running robocopy to mirror a directory from one drive to another. Robocopy detected the file in use and would stall for 30 seconds then retry to copy the file. Since I didn’t specify how many times to retry, the default was one million times. How’s that for bringing a 450GB copy operation to a standing halt! Since this job was over 50 percent complete, I didn’t want to start it over – so the question was: How do I close this file in use?

The answer was a Sysinternals program called handle.exe. When running handle with no switches, it will print out every process and the files opened by it to the screen. This will typically be more information than the default cmd window buffer will handle, thus part of the information will scroll off the window. The easist way to deal with this issue is to pipe the output to a file and then search the text file. Here is an example:

C:\temp>handle > handle.txt

Now open handle.txt in notepad and search for (in my case): ~$Weekly Sales Report.xlsx.

The search yielded the following (edited for brevity):

------------------------------------------------------------------------------
System pid: 4 NT AUTHORITY\SYSTEM
395C: File  (R--)   E:\Personals\Joe.Didley\Reports\~$Weekly Sales Report.xlsx

So now I know that the process id ’4′ had the file open with an assigned id of ’395C’.

The command to close this particular file handle is:

C:\temp>handle -c 395C -p 4

Handle v3.45
Copyright (C) 1997-2011 Mark Russinovich
Sysinternals - www.sysinternals.com

395C: File  (R--)   E:\Personals\Joe.Didley\Reports\~$Weekly Sales Report.xlsx
Close handle 395C in System (PID 4)? (y/n) y

Handle closed.

Success! After the file handle was closed, robocopy automatically continued since the file was no longer in use.

In this example, we will use a very nice tool called Expect.  Expect has traditionally been run on Unix variants, but has also been ported to Windows.  Activestate, the company known for Perl on the Windows platform, also offers TCL for Windows – which includes Expect.  This particular article will cover the program running on the Linux platform, with the possibility of revisiting at a later date to explore whether we can run the same processes in Windows.

In today’s times, even if your a full blown Windows user, there are very easy ways to add Linux into your engineering toolbox.  This is most commonly done using Virtual technology, which is offered by multiple vendors.  The more common scenarios are to download a free ‘player’, such as the one provided by vmWare. Once you have an installed VM player, you can proceed by building a basic Linux machine from scratch (which will run on top of your Windows platform), or just download a pre-built ‘appliance’ from the vmWare website. You can easily download the latest and greatest versions of Linux, ready to run, by copying an image to your workstation, hit play on the vmPlayer, login and your ready to work!  It really is that easy!

First, let’s start with a simple expect script and then gradually move into something a little more flexible. For an Operating System, I am using Ubuntu 10.10 Server Edition. The Server Edition just installs the minimum requirements to run a linux machine with basic tools. There is no GUI in the installation, so everything is done at a command line. This keeps the footprint small, which is especially good for running inside a virtual machine like I am doing.

Ok, I am logged into the Linux machine and at a command prompt. In this example, we are going to create a very simple expect script to log into a Cisco router, that is pre-configured to allow a username and password only. After a sucessful login, we will immediately be in priviledged mode. If this is not the way your test device is setup, don’t worry – I will show you how to modify the script, following this example. The script itself contains many comments (lines preceded with the ‘#’ character), which explains what the following line accomplishes.

First, let’s create the script by typing the following command:

root@ubuntu:~/util# vi 1.exp

Once in the vi editor, press ‘i‘ to insert characters and type or paste the following commands: Note: To try this on an actual device, replace the IP address shown below (192.168.1.1) with a valid device address in your network. Also adjust the username and password (admin/cisco) as necessary for your environment.

#!/usr/bin/expect -f
#Tells interpreter where the expect program is located.  This may need adjusting according to
#your specific environment.  Type ' which expect ' (without quotes) at a command prompt
#to find where it is located on your system and adjust the following line accordingly.
#
#
#Use the built in telnet program to connect to an IP and port number
spawn telnet 192.168.1.1 23
#
#The first thing we should see is a User Name prompt
expect "User Name:"
#
#Send a valid username to the device
send "admin\n"
#
#The next thing we should see is a Password prompt
expect "Password:"
#
#Send a vaild password to the device
send "cisco\n"
#
#If the device automatically assigns us to a priviledged level after successful logon,
#then we should be at an enable prompt
expect "#"
#
#Tell the device to turn off paging
send "term length 0\n"
#
#After each command issued at the enable prompt, we expect the enable prompt again to tell us the
#command has executed and is ready for another command
expect "#"
#
#Show us the running configuration on the screen
send "show run\n"
#
#The interact command is part of the expect script, which tells the script to hand off control to the user.
#This will allow you to continue to stay in the device for issuing future commands, instead of just closing
#the session after finishing running all the commands.
interact

Once these commands have been typed, press ESC key to exit out of insert mode. Then press ‘:wq‘ to write to the file 1.exp and exit the vi editor.

If you test device requires an enable password, use this script instead (with the previous mentioned modifications):

#!/usr/bin/expect -f
#Tells interpreter where the expect program is located.  This may need adjusting according to
#your specific environment.  Type ' which expect ' (without quotes) at a command prompt
#to find where it is located on your system and adjust the following line accordingly.
#
#
#Use the built in telnet program to connect to an IP and port number
spawn telnet 192.168.1.1 23
#
#The first thing we should see is a User Name prompt
expect "User Name:"
#
#Send a valid username to the device
send "admin\n"
#
#The next thing we should see is a Password prompt
expect "Password:"
#
#Send a vaild password to the device
send "cisco\n"
#
#If the device requires us to enter an enable password, then we should currently be at a
#non-privileged prompt
expect ">"
#
#Send the command to enter enable mode
send "enable\n"
#
#We should see a prompt asking for the enable password
expect "Password:"
#
#Send the enable password
send "supercisco\n"
#We should be in privileged mode now reflected by a hash prompt
expect "#"
#
#Tell the device to turn off paging
send "term length 0\n"
#
#After each command issued at the enable prompt, we expect the enable prompt again to tell us the
#command has executed and is ready for another command
expect "#"
#
#Show us the running configuration on the screen
send "show run\n"
#
#The interact command is part of the expect script, which tells the script to hand off control to the user.
#This will allow you to continue to stay in the device for issuing future commands, instead of just closing
#the session after finishing running all the commands.
interact

Now, it is time to run our test script:

root@ubuntu:~/util# expect 1.exp

Here is a sample output:

root@ubuntu:~/util# expect 1.exp
spawn telnet 192.168.1.1 23
Trying 192.168.1.1...
Connected to 192.168.1.1.
Escape character is '^]'.

User Name:admin
Password:*****

Router#term length 0
Router#show run
Building configuration...

Current configuration : 3832 bytes
!
version 12.3
service timestamps debug uptime
service timestamps log uptime
service password-encryption
!
hostname Router
!
boot-start-marker
boot-end-marker
!
!
ip subnet-zero
!
!
ip cef
no ip domain lookup
!
!
!
!
!
!
!
!
!
!
!
!
!
username admin privilege 15 secret ****
!
!
!
!
!
!
!
interface FastEthernet0/0
 ip address 192.168.1.1 255.255.255.0
 duplex auto
 speed auto
!
interface Serial0/0
 no ip address
 shutdown
!
interface FastEthernet0/1
 no ip address
 shutdown
 duplex auto
 speed auto
!
interface Serial0/1
 no ip address
 shutdown
!
no ip http server
no ip http secure-server
ip classless
!
!
!
!
!
!
!
!
!
!
!
line con 0
line aux 0
line vty 0 4
!
!
end

Router#

At the end of the script, we are left at the command prompt, so that we may continue interacting with the router.

In the next article, we will take the script to Version 2 (and beyond). Future enhancements include creating a separate file for all the devices and credentials, ability to use telnet or ssh for the connection, copy configurations from different vendors hardware.