A Solution for Skype for Business 2015 “Response groups may stop receiving calls after you run the Enable-CsTopology command”

Response group may stop receiving calls after you run the enable-CsTopology command

 

Good morning to everyone,

I have the first new ground in some time, for a Response group issue, I found in 2015, documented in 2016 and finally found it had been committed to KB4015898

Or Just SFB CU5!

 

I wrote this Piece on Response groups not ringing available agents, when the group members were in modes like longest Idle.

I said the only remedy available was to rebuild the response group and you may (or not) get a different result. Another viable step is

to “Use Attendant Mode.” This was my shtick in 2016.

 

So it turns out there had been an issue dating back 2015. I had personally run into this issue several times. It turns out somthing

you may do every day, turns out to be a “No No”. So if you have a need for “Re-Publishing” the topology Frequently, and you

have response groups, that may no longer be a good Idea, Even if there is a patch in place.

 

Just be aware, and make sure you update to CU5 or higher.

 

 

“Response group may stop receiving calls after you run the enable-CsTopology command

 

This needs to be installed on any server where you have installed the Skype for business. Back up your

Response groups for CU5.

Louis

 

Advertisements

Skype For Business Certificate Encryption Agorithm RSASSA-PSS is not supported in Skype for Business

 

I ran into this issue for the first time and just wanted to document this. It was a bit of a surprise, but documented is the fact

that Skye For Business is not supporting Current Encryption Methodology RSASSA-PSS. This was a real snake bite. Having

been involved in many deployments, I never ran into this issue. so a few point on this:

  • From Skype 4B stand point, look at this as a Certificate authority problem
    • besides failing calls you have
    • Bad certificate endpoint audits
    • bad chain audits
  • SHA1RSA is being phased out like this year was the deadline as I recall
  • You would expect SFB to have moved to more modern encryption
  • SHA2RSA is apparently not supported either but does support RSAPSS (which MS documentation says is supported)
  • What they really mean is SHA1RSA is an absolute requirement and has not changed as of 10.2017

So for the mean time this means you are more and more likely to find this occurring, until there is a breaking point, and my article will become helpful. I will try to find some errors to post to give you some terms to look up soon.

 

I have it on my desk but i am out of time. But Early waning. The New RSASSA-PSS is not currently supported by SFB.

Yeah RSASSA-PSS is not supported signature algorithm for Lync/SFB servers, this information has been provided in the following article, https://technet.microsoft.com/en-us/library/gg398066(v=ocs.15).aspx

Like it. This 2013 article is what Microsoft Passed on for Skype for Business 2013.

L

Jetstress – Too Many IOPS? Andrew Higginbotham

Hello all,

This is a shout out at my Friend Andrew Higginbotham. This man is a multi-MVP and MCM in Exchange Server. He penned an article about Jet Stress, Which is very useful.

The issue is Page Fault Stalls/sec and the subject is SSD Solid State Drives.

I admit to not spending my time in Jet stress, as I don’t work on design elements as much as I do Skype. Andrew has come to my rescue on a few Design issues and Jet Stress on more than one occasion.

It turns out you should read this if you using SSD Flash Drives and Jet Stress= Here

This quick reference in my Blog is to support Andrews Blog and Recommend you read everything he writes. He is truly one of the best Exchange Persons around the US neighborhood.

Andrew thanks for your time on this case. I hate not being the expert but I am proud to work on a team with such strengths. I am just glad to be part of a team of individuals whose strengths compliment each other.

https://exchangemaster.wordpress.com/2017/07/12/jetstress-too-many-iops/

 

Louis

Edge Replication Status is false and the Last Update Creation time stops updating for command get-csmanagementstorereplicationstatus

When it comes to Edge Replication checking, this looks like a false positive below. But I know we all like to see true. So you see where the date says 6/22? That change means the last status report was a few month’s earlier. That missing update creation, is possibly saying the replication is not working. this is not hard to fix, so lets fix it!

 

Perform the steps below:

  1. Go to the Front end server and open Skype Management Shell
  2. Run the command Export-CsConfiguration –Filename C:\filename.zip
  3. Copy the file to the Edge Server.
  4. Open the Skype for Business Deployment Wizard
  5. Choose Install Or Update Skype for Business Server System
  6. Choose install local configuration store.
  7. Browse to the file and finish the wizard.
  8. You can restart the Edge Server or just wait several minutes.
  9. If this fails, you just need to restart the SFB replication service on the FE and Edge Servers.

 

This is the point at which you browse to the configuration Zip file. Its Step 7.

I hope this helps your issue. I have seen this just stop refreshing and this step normally fixes the issue in my experience.

 

Louis

Getting Accurate Latency from Dynamically expanding Hyper-V virtual Machine Disks

 

This Article is about the tool called Hyper-V Performance Monitor Tool (PowerShell)

you can download it from the tech net article down the page, or use the link above.

So Hyper-V gets thrown around loosely these days, when you talk about Virtualization or Performance Tuning, or Planning or any other aspect of the product life-cycle of a new Host Deployment.

Over the last few years, we have made rapid changes from Physical Host Machines for production work loads, to these Virtual monstrosities, that now host our whole company.

Along with this change, you may recall that Early Hyper-V documentation has gently let us know that monitoring inside the Virtual Machine was not going to give us the parity results to the Physical counters, depending on configuration.  This is for a few reasons, which are beyond this articles scope. However, I would like to shine light on this, so more people can think differently about their Virtual performance.

The most common measure of how well a server is performing is Latency in Milliseconds. Every one is most concerned with how much latency is in the storage system. Perhaps with good reason. The San storage vendors can perform so fast now days, that you can throw the empire state building at a server, and the Latency is less then 10 Milliseconds (MS). Or is it throughput?

To be clear, we are interested more in Latency then throughput. Latency should be minimized, and throughput will generally increase.

Can I make a case that Counters are not reliable?

Well let me tell your actual Latency is not as easily attained as you would think. If I told you that your tests are lying to you, would you believe me? Lets say your in shock. Without knowing your design, my generalized answer has a very high chance of being correct.  The issue is that you have a SAN and you are tying to get latency by measuring responses to a file, that go though 4 different filters, and then has to wait until it gets queued to a disk subsystem, that is always expanding. I promise you; your numbers are incorrect.

If you cannot quantify how latency and throughput are different, but related, then I would say you should not stop reading.

Storage Latency of VM guests

There are many problems with calculating storage latency, but disk is the model we are going to use to illustrate how tricky it can be to find out how your VM is performing.

The most common approach to gaining latency information is to use a command line tool. Normally the tool will work fine. The model breaks down when the Disk itself is changing, along with the Ram and processor availability. The bottom line is a Virtual machine may lie to you about resource numbers at any given time.  Add to the mix, that the clock cycle is a weakness in any virtualization platform. That means that the calculation of time itself can result In poor results based on good math with bad numbers.

There are a crowd of you who say that is bull. Well all I can say is; don’t read this and good luck solving your latency issues.

Let me try to list some areas where the numbers may go awry. I am just making a one line explanation with a link, so you can read more. I don’t want this to be about the problems. Below, I talk more about the solution. Read more if you have a specific issue:

 

I could keep going. Do you get the feeling there are a ton of variables that change how storage latencies should be calculated?

From my experience, I have found that every set of servers are their own data set of network behavior. There are some basic assumptions I found to pass along to Admins who want to find out latency of Virtual machines.

Guidelines for VM latency Study

Who to Blame

So again, the basic message is that the calculation of latencies is totally based on the sum of the deployment factors. In one data center you may find under reporting, and the other Over Reporting. Support agents do not have the Onus to prove why one is slower then the other. We will have to look at your Design and deploy and try to make a story of things we can identify. it is not likely we will find that moment where the Deployment deviated from your Baseline storage latency measurements. We offer Best effort, but encourage you to strip down your deployment to make a core Baseline latency for a Dynamically expanding VM. All Vms will compare to that one. We go from there.

Using the Stop Gap solutions for Monitoring Virtual Machines

SO it was just a few years ago, this issue with VM monitoring was not easily remedied. You could certainly use the Perfmon counters to get VM stats. But Customers just want to run Disk Speed or SQLIO, and get an output to look at. This did not exist for quite some time. Thankfully there is a script out there, that will now carve out some parity to those tools. the link is at the TechNet Gallery:

Gallery.TechNet.Com

Hyper-V Performance Monitor Tool (PowerShell)

Below is the walk through of the basic performance collection.

you Just run the Script from an Admin PowerShell. There are a few ways to run it:

.\Monitor-HyperVGuestPerformance.ps1

### export data to csv via GUI, defaults to current dir
.\Monitor-HyperVGuestPerformance.ps1 -ExportToCsv

### retrieve data as PSobjects, great for parsing and logging, -name parameter is optional, defaults to automatic discovery
.\Monitor-HyperVGuestPerformance.ps1 -PSobjects

### specify host and interval/samples manually
.\Monitor-HyperVGuestPerformance.ps1  -Name host1,host2 -PSobjects -Interval 2 -MaxSamples 5
### accepts pipeline input
‘Host1′,’Host2’
| .\Monitor-HyperVGuestPerformance.ps1 -PSobjects

### Log to SQL server with Write-ObjectToSQL , this example uses SQL auth
.\Monitor-HyperVGuestPerformance.ps1 –PSobjects  |  Write-ObjectToSQL –TableName table –Database db -Server server –credential (get-credential)

 

image

If the domain connection fails, it tries for a Local connection:

 

image

In my case, I ran the tool on the Host, and this GUI below popped up. All I did was hit monitor, and I got an export vm_perfmon_stats file. This file can be used to find your latency.

Untitled

While this method may not be pretty, It does follow the rules for Hyper-V guest. The main purpose for this tool would be to use instead of SQLIO or DISK SPEED. tools like these should be used for hardware testing. A Hyper-V Server, running on ISCSI shared storage, with two VHDX files attached, is likely going to come back with Erroneous Latencies. This may not be perfect, but I do believe you will see a consistent result that is not a totally unbelievable number.

See I changed the Sample and interval:

Untitled

And I get a time-frame to wait for the test results:

image

Find the Link at the Microsoft TechNet Center. Thank you for taking the time to Read about Storage Performance for Hyper-V virtual Machines

I hope this helps in your Baseline Studies.

The result is a nice little Excel Display of the data, that I cleaned up a little with colors, to the Excel Fields.

 

image

Louis

Using Jperf Graphical Interface for Iperf, for basic network testing and Diskspeed or Dskspeed for storage testing.

 

Late night yawl.

Its 11:52 PM on 3/28/2017. It has been pouring in Edmond for a few hours now. For some reason I am not sleepy yet. I thought I would spend a few minutes putting together how to use a tool I was interest in.

Jperf is a script that runs on windows, and takes the place of IPERF.This little GUI should do most of the things you need, from a two windows machines. If you want to do it on Linux, then you can just use the Iperf command line.

Half the battle for Jperf is just finding the Bits. They are located here:

JPERF Download

There is also a copy of DiskSPeed or DSKSPEED.

The concept of running Jperf is simple:

  • Java must be running on both machines
  • Unzip the Jperf Folder to the Desktop
  • Use PowerShell 5 to run the Batch file on both machines
  • The server just sends data to the test subject, so to start the server, you just select the Server Radio button and hit Run IPERF in the upper right hand corner.
  • This will look like this:

 

1

 

and to run the script, all you had to do was run a batch file called Jperf.bat:

 

2

Now after starting the Jperf on the Server machine, you simply run the same batch file, on the machine that is going to be the client. Choose the client radio button, the Server Port, and Server IP. You have other choices, but you can test various packet configurations after you have showed that the two applications are doing the basic monitoring succeeding.

 

SO now that you have both GUIs up, chose run IPERF on the server, then on the client. you should see some graphing immediately.

3

The tool is nice to just show how the network is working between two points, with various network conditions, that you can control.

If you look at the enclosed User Guide, you will find where it calls out a baseline number you can use for making comparisons  the average file transfer size and the time for that average transfer, in the last line of the test is what I am talking about (see figure below)

These numbers you want to take note of. you can use this as your baseline, to compare other situations or other iterations of the test.

4

That’s IT!

Good luck and I hope this helps to explain how you can use IPERF with a basic GUI and provide good troubleshooting  information The tool is called Jperf and It uses Iperf. Its all in the download folder.

Next Time

Watch out next time when I will review the replacement for SQLIO. The tool is called DSKSPD or Disk Speed. We will talk about how  “dsksped test not valid for hyper-v virtual machine storage” Hyper-v performance is not as easy as just using Disk SPeed

MS even states that Dynamically expanding disks should not be used for production. That is also where counters start to come back with irrational results.

Here is something to read on Hyper-v Storage optimization, until I have time to write the next article

Auto-Attendant announces there has been a System Error! and hangs up!

 

Hello All. I had a support case where the answer was right in front of me but I didn’t initially put my hand right on the problem. Actually there is two incidents and the customer was the same. Let me share both stories with you.

The first issue was an auto-Attendant who failed to use voice to interface with a user. Instead It screams out System error! In fact the error simply said there was a system error and hung up. It was very nasty and there was no definitive help in the event log or anywhere else.

The work around plan to fix this was to simply upgrade the Exchange CU and the .NET Framework version. As it turns out, even that would not have helped this situation.

So if your environment is just one Gateway, one Dial Plan, one Auto attendant and one Hunt group, there is a good chance you wont have this problem. However, that may also be the perfect situation for the problem. The issue looks like the product of recreating the Dial plan multiple times. Below is a good result. But what if the results were the same for three lines in a row? If you only have one dial plan, this would be something to be alarmed about!

image

Not so much the picture above, but the one below:

So with multiple hunt groups per dial plan, the search behavior changes and you may get an un documented failure. Not so much because its a problem failure, but

because, often times, documentation is written in terms of what some application will do, not what it wont do. So word to the wise, check your hunt hunt groups.

I saw the problem when you have multiple hunt groups and only one dial plan. I know there is also only one Dial Plan. But I think you can see how a seach outside of the local dial plan could render a problem.

When you run the get-UMHuntGroup command, You will see the Hunt group displayed, one for every Dial plan. This is an area where you may have old hunt groups pile up for dial plans you created and deleted.

Its fine to have old hunt groups, but if they have the same name, there could be a system error.

The system error occurs, If you have a ghost Hunt group object and:

the name of the hunt group matches the one your using for your production dial plan. and you have the “in the entire organization” checked in the address book and operator access setting in the auto attendant setting.

image

Changing the radio button to this dial plan only, protects the Auto Attendant for issues with other hunt groups, or dial plans. This is a lesson learned for me! My default setting is not this dial plan only! Such an easy thing to miss!

This seems more like a corner case then a call driver, but I have to mention it, as I am sure we will see it again! Have a great roman holiday!

 

Louis