Thursday, June 30, 2016

Skype for Business server 2015 patching guide

Skype for Business Patching Guide


Things have changed now for update process of Skype for Business server than legacy Lync 2013 upgrade process.
It has been quite easier now for administrators to patch S4B servers and also there are less decision points now.
The only thing that needs to check is Fabric state for front end servers. If fabric state is Up, services on one server can be failover to other servers in pool.
Once the server is taken offline by failing over to pool, server can be upgraded. After successful upgrade, services on the server should be failback from pool to upgraded server.
Upgrade process for other services servers is same as Lync server 2013 upgrade process.



Phase 1     Front-End and director Servers


1.0     Checking The Server


Check each Skype for Business server in Enterprise Pool for Below points.

1.1    Verify Event viewer and Pending Reboots

Event ID 1029 will be logged in the application log if an application has been installed that requires a restart.

Run Below Commands in Powershell to view if there is pending reboot on Server

$rebootreqevents = get-eventlog -log application |where {$_.instanceID -eq "1029"}
$rebootreqevents

Check the first entries date and time. In this case it is Jan 11 20:32

Now run the following commands from powershell


$lastboot = Get-WmiObject -Class Win32_OperatingSystem
write-host "Last Server Reboot:`t" $lastboot.ConvertToDateTime($lastboot.LastBootUpTime)

You will get the following output
Please make sure Server is rebooted after last pending Reboot date.

1.2    Copy Skype for business Update Installer on local drive.

Copy the patches that have been downloaded to each server to be patched.
The core reason for this logic is that then during each install a set of logs per server will get generated when applying the patch, if the installer is run from a central share the logs will be overwritten each time and will make it difficult to troubleshoot.

1.3 Check Fabric State of Front end server

As upgrade process for Skype for business has been changed, while upgrading each FE server, FE server should be failover to other servers in pool.
So Fabric state should be up before we start upgrade process.
Run below command on FE server to check fabric state
Get-CsPoolFabricState -PoolFqdn <Pool FQDN name>

Sample Output: Things to check

To verify the Fabric to be up, check parameters under Pool Fabric and services summary. Health of all servers should be Ok and their status should be UP.
Also, under Pool all server and services summary, there has to be one primary for each FE server.
If this cmdlet shows any missing replicas, then run the following cmdlet to recover the pool before you apply any patches:

Reset-CsPoolRegistrarState -ResetType QuorumLossRecovery


This command will take up to approx. 30 minutes for all services to recover the pool fabric. 

Have some coffee and wait for fabric to be recovered, it is not recommended to run this command again before 30  minutes.



1.4 Failing over server to be upgraded

While doing upgrade on a server, server should be taken down by failing over to other servers in pool; provided that Fabric should be UP.
To failover the FE server, run following command in PowerShell:
Invoke-CSComputerFailover  -computername <FE server to be upgraded>
Sample Output:
 
Command will not give any output, after this command is completed, check all Lync services on server. All services should be in stopped mode.
This cmdlet moves all services to other Front End Servers in the pool, and takes this server offline.

1.5 Run SkypeUpdateInstaller to update server


Run the locally copied Skypeupdateinstaller.exe.
You will see prompt like shown below

When you are ready to start the patching process, Click Install Updates.
This will begin the patch process which is automated from here. Monitor the deployment to ensure all goes successfully. Once the updates have been installed, restart the server if prompted to do so.
Verify all components have been updated by running skypeupdateinstaller.exe again.
Notice the column Latest Installed and observe which patches have a Green Check Circle (i.e. it has the latest patch installed for that component) or a Red X Circle (i.e. it does not have the latest patch installed for that component).


1.6 Failing back the server which is upgraded


After server is upgraded, to move services on this server gain, following command should be run on server:
Invoke-cscomputerfailback –computername <server name which is upgraded>
This services will be returned to this server.
Check if all Lync services to be up and running by running Get-CSWindowsServices.


Repeat all the steps for all the servers in the pools.
Note: check Fabric state before taking server offline by failing over.


Phase 2    Backend Servers


Note that If a Back End Server is down for at least 30 minutes while you are upgrading it, users may then go into resiliency mode. When the upgrade is finished and the Back End Servers has again connected with the Front End Servers in the pool, users are returned to full functionality. If the upgrade takes less than 30 minutes, users will not be affected.

2.1 Upgrading backend Servers in Lync enterprise edition pool

We will take a case where SQL database are mirrored.

2.1.1 verifying Database mirror state

Before starting update, make sure all the Database should be principle on primary server.
To check Database mirror state, use following command:
  1. Get-CSDatabaseMirrorState –poolfqdn <Pool FQDN> -databasetype user
  2. Get-CSDatabaseMirrorState –poolfqdn <Pool FQDN> -databasetype application
  3. Get-CSDatabaseMirrorState –poolfqdn <Pool FQDN> -databasetype centralmgmt

Sample Output
In all three outputs, you need to check whether StateOnPrimary is principal.
If any of database seems to Mirror for StateOnPrimary, then you need to failover database to primary.
You failover database to primary, use following command:
  1. Invoke-CSDatabaseFailover –poolfqdn <Pool FQDN> -newprincipal primary -databasetype user
  2. Invoke-CSDatabaseFailover –poolfqdn <Pool FQDN> -newprincipal primary -databasetype user
  3. Invoke-CSDatabaseFailover –poolfqdn <Pool FQDN> -newprincipal primary -databasetype user


2.1.2 Updating Backend database

Apply the changes made to SQL database by running following command in Lync Management shell
  1. If this is an Enterprise Edition Back End Server and there are no collocated databases on this server, such as Archiving or Monitoring databases, then type the following at a command line:
Install-CsDatabase -Update -ConfiguredDatabases -SqlServerFqdn <SQL Server FQDN>
  1. If this is an Enterprise Edition Back End Server and there are collocated databases on this server, then type the following at a command line:
Install-CsDatabase -Update -ConfiguredDatabases -SqlServerFqdn <SQL Server FQDN>  -ExcludeCollocatedStores






2.2    Verifying The Database Updates


Check each server for versions to be up to date by running following command on Front-End Server by replacing server name with each backend server.

Test-CsDatabase -ConfiguredDatabases -SqlServerFqdn servername.domain.com | FT DatabaseName, ExpectedVersion, Installedversion

Sample Output: -




Phase 3    Director, Monitoring/Archiving, Edge, Mediation Servers


1.0    Checking The Server

Check each Lync server in Enterprise Pool for Below points.

1.1    Verify Event viewer and Pending Reboots

Event ID 1029 will be logged in the application log if an application has been installed that requires a restart.

Run Below Commands in Powershell to view if there is pending reboot on Server

$rebootreqevents = get-eventlog -log application |where {$_.instanceID -eq "1029"}
$rebootreqevents


Check the first entries date and time. In this case it is Jan 11 20:32

Now run the following commands from powershell


$lastboot = Get-WmiObject -Class Win32_OperatingSystem
write-host "Last Server Reboot:`t" $lastboot.ConvertToDateTime($lastboot.LastBootUpTime)

You will get the following output
Please make sure Server is rebooted after last pending Reboot date.

1.2    Copy Lync Update Installer on local drive.

Copy the patches that have been downloaded to each server to be patched.
The core reason for this logic is that then during each install a set of logs per server will get generated when applying the patch, if the installer is run from a central share the logs will be overwritten each time and will make it difficult to troubleshoot.

1.3     Check Lync services

Check all Lync/SQL services are up and running each Server in one pool.
Run Following Command in PowerShell, run as admin

Run-CsWindowsService

If services are up and running on all servers in one pool, Open Lync update installer on one of the server,

2.0    Updating the Server

Navigate to the local copy of the patch and double click Lync update installer
In our case, Lync update installer have been placed in D:\Lync Updates. (IN Geo)

Run Installer, following screen should appear


When you are ready to start the patching process, Click Install Updates.
Once the updates have been installed, restart the server if prompted to do so.

3.0    Making sure if Updates are installed

After restart, check if all services are up and running by running following command in PowerShell.
Get-CsWindowsService

If all services are running,
Run Lync update installer again and check if last column has all greens check circles.
Then go for Second server in each pool/domain.

4.0 Installing update on Monitoring /archiving database

Apply the changes made to SQL database by running following command in Lync Management shell
Install-CsDatabase -Update -ConfiguredDatabases -SqlServerFqdn <SQL Server FQDN>

Where SQLserverfqdn is Monitoring SQL server’s fqdn.

5.0 Verifying Database update changes

Check for versions to be up to date by running following command on Front-End Server by replacing server name with monitoring server.

Test-CsDatabase -ConfiguredDatabases -SqlServerFqdn servername.domain.com

Sample Output: -









---------------------------------------------------------------END---------------------------------------------------------------------