Running The Population Wizard

The first thing we’re going to want to do once the Connections applications successfully install is test them by logging in as a regular user.  To do that the user must have a profile in the PEOPLEDB or Connections will throw an error and the easiest way to ensure that exists is to run the Population Wizard.

The Population Wizard is a simple tool that asks a series of questions such as where is your LDAP directory, where are your DB2 databases, and where is TDI and uses those answers to run a one off script using TDI to import all users into Connections, creating them a profile along the way.  It’s designed to work in only one direction (you can pull information from LDAP to databases only, not the other way around).

For a test environment or a very small manually-maintained environment you could run the PopulationWizard to update the databases whenever you want but for production it’s not really workable.  However, this is just to ensure we have some data in place that we can use for testing.  We will work with the custom TDI scripts later to fine tune what we want.

To run the population wizard look in the “Wizards” directory from the ConnectionsWizards download.  The file will be called populationWizard.sh (or populationWizard.bat).

skitch.6

skitch.5

The JDBC drive path must contain the drivers for your database platform.  If you aren’t running TDI on the same server as your databases you will need to copy the drivers from the database server to the TDI server to reference them here.

The User ID is the account granted full rights to the PEOPLEDB database.  This can often be a custom account and not (as in this case) the instance owner.

skitch.4

Here we tell the wizard where our LDAP server is and how to connect to it.  If you are using a secure port such as 636 make sure you check the box for “Use SSL communication”

skitch.3

These are the bind credentials to login and query LDAP.  They aren’t stored anywhere in this activity or used for any purpose other than running this one-off wizard so any credentials would do.

skitch.2

This shows the sample mapping of LDAP attributes to database fields. For example “mail” in LDAP will map to the field “Office Email”.  On this screen we can choose what attributes to map where and even if we want to map attributes at all.  Anything we don’t map won’t be populated with data and will appear as empty in Connections.

Once the populationWizard completes it should report that it imported your users.  To do this it wrote instructions to properties files and ran script files stored in the location:

/Wizards/TDIPopulation/linux/TDI –

The files it wrote our instructions to are:

profiles_tdi.properties
solutions.properties
mapdb_repos_from_source.properties

The script files it ran were:

collect_dns.sh
populate_from_dn_file.sh

The activity is recorded in /Wizards/TDIPopulation/linux/TDI/Logs.

We’ll come back to this later when we start customising our syncing activity.

Creating The Databases For Connections

Before we can install the Connections applications we must create the databases they will use.  The database server can be DB2, SQL or Oracle but the licensing for DB2 is part of your Connections licensing whereas SQL and Oracle would need to be separately licensed.  I tend to use whatever server the customer feels most comfortable supporting, if they are a big SQL or Oracle house I’ll use that.  For our purposes, and if the customer has no preference I like to use DB2.  One note about high availability, the license for DB2 includes the rights to use active/passive HADR for DB2  which means creating two servers which sync with each other but with only one active at a time. To make the active server passive and make the passive server active requires a manual switchover and will entail some amount of downtime.  For additional license cost DB2 offer a full HADR active/active solution but I won’t be discussing that on this blog.

We already installed DB2 so now we just need to run the DBWizard to create the databases.  It’s important to use the db2inst1 account we created during install as “owner” of the DB2 instance (on Windows it’s probably called db2admin) to create the databases. This ensures that the db2inst1 has the right security access to the databases and that later, when the Connections applications attempt to write to the databases, everything works.

Now I can login as db2inst1.  Don’t use “su” as that can throw errors, always login to the server as the db2inst1 (or in Windows db2admin) account.

Setting The Environment

The first thing we should do is make sure the instance of DB2 can run all the Connections databases.  Run the command

db2 update dbm cfg using numdb 20 to allow up to 20 databases to run in this single instance.  That’s a high number but in a reasonably small environment (less than 1000 concurrent users say) it’s fine. 

We also need to set the DB2 instance to use unicode before Connections installs, we can do that whilst we’re here by typing

db2set db2codepage=1208

Then stop and start DB2 to make both the above changes take effect

db2stop
db2start

Creating The Databases

IBM released Day 1 fixes for the DBWizards and you need to find and use these not the ones that shipped with 5.5.  They are on fixcentral and their filenames are

Linux: 5.5.0.0-IC-D1-DBWizard-LO87408_lin_aix.tar
Windows: 5.5.0.0-IC-D1-DBWizard-LO87408_Windows.zip

Using my root account I extract the tar file to a directory like this

mkdir /home2/db2inst1/DBWizard
cd /home/db2inst1/DBWizard
tar -xvf /opt/Software/5.5.0.0-IC-D1-DBWizard-LO87408_lin_aix.tar chown -R db2inst1 * (this recursively makes the file owner of all files in that directory db2inst1)

skitch.9

Launch the ./dbWizard.sh from the terminal session.  This has a graphical interface so if you don’t have one configured you will need to create a response file and run ./db2Wizard -silent -nameofresponsefile insteaad

skitch.8

skitch.7

The same dbWizard is used to create, update and delete databases, it can also just generate the SQL commands to enable you to manually do these activities yourself.  Depending on the complexity of the enviornment and / or if the Wizard starts throwing errors, I often take the SQL commands and manually run them myself so I can monitor and adjust them if necessary.

skitch.6

The second screen asks for (and defaults to) the location of my DB2 install as well as the account that owns that DB instance.  I am running the Wizard using that account to ensure there are no security issues with permissions on the newly created databases.

skitch.5

I have chosen not to create either the Cognos or the Connections Content Manager databases at this time beacuse I will not yet be installing those features.  When I come to install them later I will re-run this Wizard (or the latest version of this Wizard) and create them then.

skitch.2

The final screen shows all the commands that the Wizard is about to run to create each of the databases. These commands point to .sql files that were extracted into the DBWizard directory and hold instructions on how to build each database for each application.  I can save those commands to a file and keep them for reference or use them manually later.

If you want to look at what they are doing, the sql files and routines are under the connections.sql directory in the extracted Wizard location.

skitch.1

The wizard will now run through creating every database and logging the activity to the /home/db2inst1/lcwizard/log/dbWizard directory with one log file for each sql routine.  I like to review the log file activity as it progresses in case there are any issues – the entire process can take anywhere from 30 minutues to 2hrs depending on the speed of your disk.

skitch

Once the Wizard has finished, try to connect to one of the newly created databases from a terminal window to confirm the database is there and db2inst1 has the right access to it.

skitch.10

For example to connect to the Activities database I use
db2 connect to opnact

Next step: configuring LDAP and populating the profiles database

Installing WebSphere

We can’t install Connections until both WebSphere and the DB2 (or Oracle or SQL) databases are in place so let’s start with WebSphere.  On previous posts we looked at installing Installation Manager (IM) and setting up a software repository to point to the WebSphere installers

skitch.21

My first check is to decide what version and fixpack I need and I do that by checking the System Requirements. These can change any time so I make a point to check them just before doing an install.   According to the current requirements Connections 5.5 needs a minimum of WebSphere Application Server Network Deployment 8.5.5 Fixpack 6. Since the requirement is a minimum I could install the most recent Fixpack 8 but for my purposes I’m sticking with Fixpack 6 today.

skitch.20

Even though at this point I’m not installing IHS I can see on the system requirements that it only supports 8.5.5 fixpack 6 of IHS.  This is something I need to remember later.

skitch23

When  launching IM and clicking “Install” I am presented with serveral products with their latest versions and options.  I have to click the checkbox “Show All Versions” to choose an earlier fixpack such as Fixpack 6

skitch.19

There are now hundreds of products and versions to choose from so to make it easier I type “network deployment” in the search bar at the top to filter the product choices (and so I don’t accidentally choose the wrong one). I then select 8.5.5.6 as the version I want to install. I haven’t downloaded 8.5.5.6, only 8.5.5 so I’m relying on IM to get the fixpack from the Passport Advantage site.

skitch.18

There are already several fixes available for 8.5.5.6 but, having checked the System Requirements to ensure there are no specifics fixes I need, I let IM choose the recommended ones only.

skitch.17

I now identify two locations, The first is the location for the shared resources directory used by IM.  This is the location for IM to store the installers, fixes and rollback files, not the location of the WebSphere program itself.

skitch.16

Now I can tell IM where to install WebSphere.  The default “WebSphere” directory will always be /IBM/WebSphere/AppServer and I would only change this if I was installing multiple WebSphere versions on the same machine (unlikely).  On Linux this defaults to /opt/IBM/WebSphere/AppServer and on Windows I try to put it on a separate drive partition from the OS such as D:\IBM\WebSphere\AppServer.  This location is your “WebSphere program directory” that is referred to in documentation.

skitch.15

These are the standard features I install for Connections.  At this point the total install size should be about 2.5GB across both directories.

skitch.14

Our confirmation screen just before the install starts shows what products will be installed into which locations.  Read the target location details at the top of this screen to confirm you have entered them correctly.

skitch.13

Our WebSphere install is now complete (love that big green tick) but now I need to create profiles to set up the servers.  Selecting OK here will launch the profile management GUI tool but this can also be launched later off the GUI menu or you can create profiles by using the /opt/IBM/WebSphere/AppServer/bin/manageprofiles.sh command

skitch.12

Since this is the first server in my deployment I need to create a new cell which will include both a deployment manager and an application server.  I often like to create the deployment manager standalone on its own machine if I’m building a HA cluster but for our purposes in Connections101 I’m just locating both the deployment manager and my first application server together.  These are the first servers on “bogart” as shown below.  If I were creating an additional Application Sever on “marlowe” then I would be creating an Application Server profile only at this point.

WebSphereExamples

 

skitch.11

Often during profile creation I would just choose the “Typical” option which is a far simpler series of choices with some default settings.  I’m choosing Advanced profile creation here to show you all the settings available to you.  The difference between “Typical” and “Advanced” is purely the ability to custom set options like ports, certificates and passwords – all of which can also be changed later after the profile already exists.

skitch.10

I always want to deploy the administrative console if I’m deploying a new cell because that’s how I’m going to manage all the servers.  If I add an existing Application server into an existing cell by federating it, its administrative console will be removed so it can be managed only by the deployment manager.

I often choose not to deploy the default applications for security reasons because it just gives me more things to have to lock down.  It can be useful for WAS troubleshooting but I rarely use it.

skitch.9

The first deployment manager on a node is usually put in a profile Dmgr01, similarly the first application server is put in the profile AppSrv01.  You can change both of these to something more meaningful if you want.  I never usually do. One thing to note in WebSphere world is to stay away from anything that could be a special character for path names or passwords.  As I mentioned on the previous post, all WAS configuration is stored in XML files and often encrypted so decrypting those special characters can fail and cause WebSphere itself to stop working.

Specific special characters to avoid for anything WebSphere related include

\/*,;:=+?{}[]<>_%^&$'”^()

but I like to avoid anything that could be problematic including accented characters.  Longer passwords can be just as effective.

skitch.8

The hostname for the server is picked up from the server’s local settings or host file. You need to make sure it’s a fully qualified name and not just “bogart”.  It also needs to be fully resolvable with an IPV4 address from itself and any other server.  The default deployment manager and application node names will be “servername”CellManager01 and “servername”Node01.  This is important because the servers will be installed with these names and a lot of nested directories will use the names.  IBM recommend your server name is no longer than 8 characters for this reason (avoiding creating very very long filepaths) but if your servername is long you can change the node names here to something shorter..

The cell name and node names cannot be changed post install and will be used to reference the servers internally.  It will always be possible to add another resolvable fully qualified hostname like “hammett.connecitons101.info” as an alias later but you can never fully remove these names.

skitch.6

When WebSphere is installed new SSL certificates are created using an IBM internal Certificate Authority and the servers’ hostnames and roles.  These certificates are usually replaced by me post install with valid SSL certificates from a public or internal provider.  If however you have a wildcard certificate for your organisation already you could add it during the install and avoid having to replace the internal IBM certificate later.  You would still need to add the trusted root though and I’m a control freak so I like to let IBM create its certificates then manually go in and replace them later.

skitch.5

The default certificates that are created (assuming you didn’t choose to replace them on the previous screen) will have keystore passwords of WebAS.  This is well known as IBM’s  default keystore password and for that reason WebSphere will see it as a security issue if you don’t change the password for keystores on a running server.

Again I prefer to let it install and then change it later but you can change it here during the install process.

skitch.4

There are three pages of ports that WebSphere will use for different services.  Unless there’s a specific reason you can’t use one of the default ports I suggest leaving these as they are (in a “Typical” install you never see these screens).  If you must change them I would suggest only incrementing them by “1” until you get to a port number you are happy with.

skitch.3

On Linux and Windows you are presented with the option to add the Deployment Manager to run as a service when the server starts.  You don’t get that option for either the nodeagent or any of the application servers which you will have to add manually after all the install work is complete.

skitch.2

Finally I get the option to create an IHS server definition.  I will need to do this at some point since we are going to put IHS in front of our Connections servers but I haven’t installed IHS yet and so I’m going to leave this for later.

That’s our WebSphere install complete.  It should take anywhere from 15 mins to an hour depending on your disk write speed.

skitch.1

Once installed, if you have a GUI, you will see IBM Websphere on the Application menu. From here you can find the profiles that were created as well as create new ones and start or stop individual servers.

skitch

The above screenshot shows me checking the install completed successfully.

First I do an “ls” to see there are both profiles there.  The system shows me Dmgr01 (the deployment manager profile) and AppSrv01 (the application server profile).  That’s good.

Now I navigate to /opt/IBM/WebSphere/AppServer/Dmgr01/bin and run ./startManager.sh (note case sensitivity).  This should start the deployment manager from within its own profile.

Once it has started I want to check it’s listening and I know the deployment manager listens on port 9043 for the Integrated Solutions Console so I type

netstat -tulvn |grep :9043 to find if that service is listening.

We now have WebSphere installed.  If you are setting up multiple servers on different boxes you would now need to go and repeat the install but this time choosing to create only an Application Server profile on each of those boxes.  Remember to keep your WAS version on every server in your cell aligned at 8.5.5.6.

Organising The Software

The first task after I’ve built the Deployment manager OS is to get all the installers and fixes downloaded.  In this environment we will have both Linux 64bit and Windows elements and so both copies of Installation Manager are downloaded for instance.

Here’s my directory structure – I use this and either scp or winscp to copy files onto other servers for installation.  It’s entirely a personal preference but I find having all the software organised into folders in a single location makes it easier to identify what I installed and ensures I don’t miss anything.

The jre was purely for using download manager on the IBM software site 🙂

Organising Software

Linux Setup And Tools

I like installing on Linux and know just enough about RHEL and SLES to get the work done that I need to do. I wouldn’t call myself an expert but I know what works for me to build Connections.  For instance there are several things I have on my checklist when deploying a Linux OS for Connections.

  1. I always deploy using supported distros.  That means for me specifically RHEL or SLES. I know people deploy using CentOS and others – I don’t.  Partly because it’s not supported so if it goes wrong I can’t expect IBM to help me and partly because I’m providing a Connections installation consultancy not Linux consultancy – when I quote work it’s to install Connections not to go down a rabbit hole of Linux workarounds.  I would feel bad charging people for Linux consultancy, there are far better Linux experts out there!
  2. I always take a subscription so I can update the OS and download any tools or libraries I need.  For this demo build which is only going to run for a few months I’m using a 30 day RHEL trial license so I can set it up properly.  Once more it’s about time. It costs a lot more in my time to manually find, download, place and install packages without subscription then a single license costs.  When you install RHEL the first thing you are going to want to do is update and add packages, with subscription you can do that in minutes rather than hours.
  3. I like to use gedit for editing XML files in place (yum install gedit) rather than vi or another text editor. The display of the structure and attributes is very clear which makes it easy for me to find content and spot typos. Connections is all about XML files
  4. I use WinSCP to copy files from a Windows machine over to a Linux one.  If I’m using my Mac I just use scp from within terminal but often I am on a remote server on a customer site and winscp is easier to deploy and use.  It’s also really easy to recursively copy entire directories like the log directories off the server to read.  Whenever I’m analysing logs, for all but the simplest reviews, I prefer to offload them from the server and review them on my workstation
  5. I use AQT (Advanced Query Tool) for opening and reviewing databases and install it on my Windows workstation.  It’s not expensive and supports all ODBC connections and a lot more. I have a blog piece on it here
  6. I install a graphical interface on Linux (usually GNOME) because I prefer working with that for things like Installation Manager and because X11 forwarding is too slow through most customer VPNs to make it workable for me.  If you’re not a seasoned Linux expert having a GUI will save you a lot of stress and make you better able to manage the environment.
  7. I use NoMachine as a remote desktop on Linux which ensures my session activity doesn’t timeout even if I go away, shut my laptop, get some sleep.  It also means other people can connect and takeover my session where I left it.
  8. I edit the /etc/hosts file and ensure that I have explicit entries for the local server, whatever hostnames it may use and all the other servers in my Connections environment with their hostnames.  I could rely on DNS to do this.  I choose to have more direct control over name resolution by using the hosts file.

So all of that is in place and now I’m ready to get started.

Step 1 Preparation and Design

First we need to consider what we are going to want (or potentially want) in our environment, even if we’re not installing now.  For instance I’m going to start by installing all the standard Connections applications but I know I will want to install IBM Docs and Cognos further down the road.

This is a small environment which I am confident will not be growing but in most circumstances I would make sure to separate the Deployment Manager server from the Application Servers to give flexibility for the architecture going forwards. Similarly I would never install IBM HTTP Server on the same instance as the Application Server or Deployment Manager if I planned to grow the environment in the future.

In our model I’ve  separated the IHS server onto a Windows 2012 R2 instance for no other reason than I wanted to show it on a separate server and I wanted to talk about both Windows and Linux deployments.

Connections101

Time To Download Take 2 – Connections 5.5

Update for Connections 5.5 – the previous post referred to Connections 5 and we will be installing 5.5 which shipped Dec 18th 2015.

There were a Day 1 fixes for Connections which must be installed. 

I usually set aside a day in my planning to find all the latest software, the latest patches and get them in place and extracted on the servers I’m using.  I like to create a “Software” directory on the deployment manager to put everything in one place and so I don’t have to keep copies on every server but that takes space. Expect to need up to 50GB for your installers. To manage space I like to delete the  extracted directories once I’ve completed the installs

None of the downloads I’ve listed below are optional and that certainly includes the fixpacks and fixes. You need all of these if you’re going to install Connections.

IBM Installation Manager

I like to download the latest 64bit version.  It will give you a warning when you install Connections that Connections itself is a 32bit product but I find the 64bit version works better regardless.  The latest version if 1.8.4 but find all versions here with links to their downloads  on Fix Central (where you’ll need to login).

WebSphere 8.5.5

Comes in three parts and you’ll need all three extracted to the same directory.

CIK2HML IBM WebSphere Application Server Network Deployment V8.5.5 (1 of 3)

CIK2IML IBM WebSphere Application Server Network Deployment V8.5.5 (2 of 3)

CIK2JML IBM WebSphere Application Server Network Deployment V8.5.5 (3 of 3)

Websphere 8.5.5 Fixpack 6

Once WebSphere is installed,  Installation Manager will connect to the IBM download sites to bring down fixes and fixpacks on demand.  To do this you need a login for your Passport account and your server needs to be able to connect outbound on port 80 either directly or via a proxy.  I find this a much simpler way of deploying updates than finding and downloading the files so I usually don’t download them in advance.

If your server isn’t going to be able to do that you will need to download the fixpack from this document

IBM HTTP Server 8.5.5

IHS is found inside the WebSphere Supplementals downloads, along with other tools that we’re not going to use here.  There are once again three separate download files and you’ll need to get all three and extract them all to their own directory.  Which I like to call WASSUPP because it makes me smile.

CIK1VML IBM WebSphere Application Server V8.5.5 Supplements (1 of 3)

CIK1WML IBM WebSphere Application Server V8.5.5 Supplements (2 of 3)

CIK1XML IBM WebSphere Application Server V8.5.5 Supplements (3 of 3)

DB2 10.5 Fixpack 5

There are a lot and I mean A LOT of DB2 products but the one I’m going to download and use is

CIXV0ML IBM DB2 Server V10.5 for Linux on AMD64 and Intel EM64T systems x64) Multilingual

For a full list of DB2 10.5 products and part numbers take a look here

DB2 10.5 Fixpack 5 (10.5.0.5) is downloadable from Fix Central via this document.  I like to download the Universal Fix Pack installer.

All download links for all DB2 version and platform fixpacks are found on this technote

Tivoli Directory Integrator 7.1.1

CZUF3ML IBM Tivoli Directory Integrator Identity Edition V7.1.1 for Linux – x86-64,

TDI Fixpack 4

The documentation and a link to the Fix Central download is here

The Fix Central homepage is here

IBM Connections Wizards

CN80DML IBM Connections V5.5 Wizard for Windows Multilingual

IBM Connections Installer

CN80AML IBM Connections V5.5 for Linux Multilingual

Day 1

Details of the upgrade strategy for the Day 1 fixes are here

Upgrade Installer (needed to install the updates) here

Day 1 Connections Fixes here

Database Wizard (Windows) here

iFix 5.5.0.0-IC-News-IFLO87487

iFix  5.5.0.0-TypeAhead-20151218

iFix  5.5.0.0-IC-Common-IFLO87469

Updated migration tool (only used for migrating from a previous install) here