Recently I had cause to investigate performance and tuning of BizTalk Server 2004, and found that although there was some useful and interesting information available, there was usually little actual guidance on how to apply even the simplest settings to your specific solution/environment.
When you take a look at the guidance available for tuning IIS, there are some reasonably simple steps that you can follow to get you started; for example you can make some simple changes to machine.config based on some simple calculations and the number of CPUs in use. I previsouly discussed this inTuning IIS - machine.config Settings.
For BizTalk Server 2004, once you look beyond the hardware options (such as scaling up the CPU(s) in use or scaling out the number of BizTalk Servers in use) I was unable to find much that was as simple to follow as these IIS guidelines.
Although I have still not yet found the magic formula, the potential for tweaking your BizTalk Server 2004 installation is there, so here are some thoughts and settings that you may find useful when looking into this yourself.
Create Separate Hosts
It is highly recommended that you create separate hosts to handle different jobs and processes within your BizTalk Server 2004 solution.
I would suggest that a good starting point is to consider separating out the following:
- Orchestrations
- Adapters
- BizTalk Host Tracking
Separating the Orchestrations from the Adapters will also help reduce resource contention when dealing with higher load, as you will have separate host processes dealing with the send/receive (adapters) and the business process (orchetsrations) for each process being handled.
You may also wish to investigate further separtion of these hosts, creating separate hosts for related Orchestrations and Adapter types. My preferred option is to provide Orchestration hosts for each of my "applications" and then a separate host for each Adapter type.
As each host runs in its own process, this gives you a more robust and flexibible solution; individual hosts can be stopped and started and a failure in one host will not automatically bring down all processes.
BizTalk Server Engine
The performance characteristics of the BizTalk Server engine can be modified by changing the following columns in the adm_ServiceClass table of the BizTalkMgmtDb database:
HighWatermark / LowWatermark
These settings determine the outbound processing rate for messages, representing the high and medium stress-level thresholds. They define the number of messages processed by BizTalk Server 2004, but not yet consumed by subscribers.
When BizTalk Server processes more messages than specified by the HighWatermark, it will stop processing messages from the message box until the number of active messages drops below the LowWatermark threshold.
HighMemorymark / LowMemorymark
These settings control the memory thresholds at which BizTalk Server 2004 starts and stops processing messages. They define the percentage of overall memory consumed and affect both inbound and outbound throughput.
When BizTalk Server memory consumption reaches the level defined by the LowMemorymark, BizTalk Server increases the stress level, and by extension the CPU usage on the host server.
If memory consumption reaches the level defined by the HighMemorymark, BizTalk Server stops processing messages until memory consumption is reduced.
BizTalk Server also stops creating new orchestrations when the memory consumption reaches the HighMemorymark. Orchestration creation is only resumed once the memory consumption reaches the LowMemorymark.
HighSessionmark / LowSessionmark
These settings determine the inbound processing rate for messages and represent high and medium stress-level thresholds.
They define the number of parallel database sessions that are persisting messages to the message box. When the number of sessions specified by the HighSessionmark is exceeded, BizTalk Server blocks incoming messages until the number of sessions decreases below the LowSessionmark
Threading Issues
With thread starvation, you may see slow performance when using the SOAP, HTTP, or WSE adapters, including situations where a large flood of messages can trigger a degradation of performance.
The BizTalk host(s) with the SOAP, HTTP or WSE Adapters running has worker threads handling queued work items and I/O threads for completed asynchronous I/O requests. Problems can occur when there are not enough free threads in the thread pools to handle the number of messages (SOAP, HTTP, or WSE requests).
A freshly started BizTalk host has default thread pool sizes that are small and therfore may not be able to handle requests made of it. The adapter must then add more worker or I/O threads to the thread pool, stopping once there are enough to fulfill the request or the maximum thread limit is reached. This can be time consuming.
If the maximum thread pool limit is not large enough to handle the sustained work load, messages must wait until a thread becomes available before they can be processed.
Minimum worker and I/O thread pool sizes should be set at a level appropriate to the initial and sustained work load.
MinIOThreads
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc.3.0{guid}]\CLR Hosting
This DWORD is the minimum number of threads, per processor, available from the CLR thread pool at any given time.
This number is automatically multiplied by the number of available CPUs and should have the same values as MinWorkerThreads. If MinWorkerThreads is configured but MinIOThreads is not, BizTalk server sets this to the value of MaxWorkerThreads.
Default 1
Recommended Maximum number of messages received at initialization + 10 percent
MinWorkerThreads
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\ Services\BTSSvc.3.0{guid}]\CLR Hosting
This DWORD is the minimum number of threads, per processor, available from the CLR thread pool at any given time.
This number is automatically multiplied by the number of available CPUs.
Default 1
Recommended Maximum number of messages received at initialization + 10 percent
The maximum worker and I/O thread pool sizes should be set to cater for sustained and peak burst loads. These loads should be defined by the business requirements and validated during stress testing. It is not recommended to define excessive thread pool sizes, as although this may resolve thread starvation it can increase context switching and this can offset the performance gained
MaxIOThreads
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc.3.0{guid}
This DWORD is the number of threads per processor that will run to handle I/O within the common language runtime or Microsoft .NET connection software.
This number is automatically multiplied by the number of available CPUs and should have the same values as MaxWorkerThreads. If MaxWorkerThreads is configured but MaxIOThreads is not, BizTalk server sets this to the value of MaxWorkerThreads.
Default: 20
MaxWorkerThreads
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc.3.0{guid}
This DWORD is the number of threads per processor that the HTTP adapter will use to process messages.
This number is automatically multiplied by the number of available CPUs.
Default: 25
A larger thread pool can put more pressure on your Biztalk Server CPUs and the SQL Server hosting the BizTalk databases. Increasing the MaxWorkerThreads value beyond 100 can have an adverse effect on the performance of SQL Server. Make sure the computer(s) installed with SQL Server can handle the additional stress before increasing the thread pool size greater than 100.
MessagingThreadPoolSize
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc.3.0{guid}
This defines the number of threads that will be used to process messages on a per processor basis on the computer running BizTalk Server.
MessagingThreadsPerCpu
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc.3.0 (Isolated Host)
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc{guid of the host} (In-process hosts)
This DWORD defines the number of threads per CPU processor that BizTalk Server uses to process incoming message batches for the isolated host or in-process host that you define it against. Larger numbers increase the CPU processor utilization in the receive host. Smaller numbers might improve performance if there is excessive context switching in the receive host.
ASP .NET Settings Related to HTTP, SOAP, or WSE Adapter Performance
In addition to thread starvation, there are several ASP.NET settings affecting SOAP, HTTP, or WSE adapter performance. These settings are made in the web.config or machine.config files. Because these adapters communicate with external Web sites or Web services, consider tuning these external components as part of overall performance review. The settings described in this section were sugegsted to be specific to BizTalk Server, but are similar to changes that can be made when tuning IIS.
The tunable attributes include maxconnection, maxWorkerThreads, minWorkerThreads, maxIOThreads, and minIOThreads. The properties that can be tuned include minFreeThreads and minLocalRequestFreeThreads.
maxconnection
Controls the maximum number of outgoing HTTP connections that you can initiate from a client to a specific IP address. This can be a bottleneck if BizTalk Server makes a large number of HTTP, SOAP, or WSE requests.
Setting the address attribute to * applies this value to all addresses. You can include multiple entries for this setting to control the maximum number of connections to specific IP addresses.
The maxconnection parameter setting applies at the AppDomain level. By default, you can create a maximum of two connections to a specific IP address from each AppDomain in your process.
Increasing maxconnection attribute increases thread pool and CPU utilization. This may mean that the thread pool settings described in the previous section may not be optimal. Test larger thread pool sizes to determine if adjustment is necessary.
This setting is not automatically multiplied by the number of available CPUs, you must do this yourself.
Default 2
Recommended 12 * Number of CPUs
maxIoThreads
Controls the maximum number of I/O threads in the .NET thread pool.
This number is automatically multiplied by the number of available CPUs.
Default 20
Recommended 100
maxWorkerThreads
Controls the maximum number of worker threads in the thread pool.
This number is then automatically multiplied by the number of available CPUs.
Default 20
Recommended 100
minWorkerThreads
Determines how many worker threads may be made available immediately to service a remote request. This can be useful when there may be sudden rise in the number of requests, such as a burst of requests from another server or from client users. This enables ASP.NET to service requests that may be suddenly filling the ASP.NET request queue.
This number is then automatically multiplied by the number of available CPUs.
Default 1
Recommended maxWorkerThreads / 2
minIoThreads
Determines how many of I/O threads may be made available immediately in the .NET thread pool.
This number is then automatically multiplied by the number of available CPUs.
Default 1
Recommended maxIoThreads / 2
minFreeThreads
Used by the worker process to queue all the incoming requests. This only applies if the number of available threads in the thread pool falls below the value for this setting.
This setting is not automatically multiplied by the number of available CPUs, you must do this yourself.
This setting effectively limits the number of requests that can run concurrently to maxWorkerThreads - minFreeThreads .
Default 8
Recommended 88 * Number of CPUs
Following the recommendation above limits the number of concurrent requests to 12 per CPU (assuming maxWorkerThreads is 100) because 100 - 88 = 12. This leaves at least 88 worker threads per CPU and 88 completion port threads per CPU available for other uses (such as for the Web service callbacks).
minLocalRequestFreeThreads
Used by the worker process to queue requests from localhost, such as requests to a local Web service. This only applies if the number of available threads in the thread pool falls below this number.
This setting is not automatically multiplied by the number of available CPUs, you must do this yourself.
This setting is similar to minFreeThreads but it only applies to localhost requests from the local computer.
Default 4
Recommended 76 * Number
Messaging Changes
HTTPBatchSize
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc.3.0\ HTTPReceive
This setting specifies the batch size that the HTTP receive adapter uses to submit requests to BizTalk Server. By default the HTTP Receive adapter waits for a batch of ten messages to accumulate in order to submit them all at once, or until a one-second time-out has elapsed.
You can decrease the latency of HTTP request/response processes by creating and setting this DWORD registry key to 1. This forces the HTTP receive adapter to submit each message as soon as it is received.
Links
The following MSDN links both have some great information (and lots of it) but you are still left to your own devices when applying these to your specific scenario:
BizTalk Server 2004: Performance Tuning for Low Latency Messaging
BizTalk Server 2004 Performance Characteristics
I also found a decent overview of a particular implementation of some of these settings, although it is not fully clear what the scenario that is being implemented is or how the setting used were decided upon:
BizTalk 2004 Tuning Recommendations
For BizTalk 2006 I found a decent overview of what can be done to help when using the Soap adapter, and much of this is also applicable to BizTalk Server 2004:
'BIZTALK > DOC' 카테고리의 다른 글
| IIS 6.0 Tuning for Performance (0) | 2009.11.23 |
|---|---|
| Tuning IIS - machine.config settings (0) | 2009.11.23 |