Ceryx: High-Availability, Hosted Exchange

In Exchange 2007 administrators will have a more robust way of managing their remote ActiveSync users with ActiveSync Mailbox Policies. This will allow administrators to enforce settings to control how users use their ActiveSync mobile devices. This means administrators have more control and security when deploying ActiveSync devices.

Below is a list of the few of the settings you can set:

Alphanumeric password required –
- Requires that the password contains both numbers and letters.

Maximum failed password attempts –
- Set the number of times a user can enter an incorrect password before the device wipes itself.

Attachments enabled –
- Enabled the downloading of email attachments.

Maximum inactivity time lock –
- Set the maximum time the device can be inactive before it locks.

WSS file access -
- Allow access to SharePoint sites.

The main benefit over Exchange 2003 in Exchange 2007 is that an administrator has the ability to set a policy on a user by user basis, whereas in Exchange 2003 policies would be set globally. In Exchange 2007 there are two ways of creating ActiveSync policies: using the Exchange Management Shell or Exchange Management Console. The management console only has the ability to set some of the settings; the Management Shell is where all other settings can be found. Also note that you do not have to specify all policy settings when creating a new policy as any policy setting that you do not set will keep the default value.

Windows Power Shell, the scripting language introduced by Microsoft last year, is a welcome addition to the admin toolbox. Command line tools experienced a decade of neglect as GUI adminstrative tools took over the landscape. “In the beginning.. ” is an entertaining yarn by Neal Stephenson about the history of the much maligned command line.
The Exchange 2007 team adopted an approach that lets us have our cake and eat it too. We now have the GUI Exchange Management Console for convenience that always shows the equivalent Exchange Management Shell cmdlets that are being used to accomplish each operation.
Get-Mailbox is an Exchange Shell cmdlet that can locate one or more mailboxes and report on various properties.

Use the Select-Object cmdlet to display just the properties you’re interested in as opposed to the default display properties. For example, showing only the warning and send limits is often desired.

You could then use the Set-Mailbox cmdlet to change the quota.

This is all fine except you may be wondering what the big deal is all about. An administrator could just as easily use the Exchange Management Console and do the same thing with less effort. The real power of Power Shell (pardon the pun) is the combination of assorted cmdlets to automate repetitive tasks.
Let’s suppose we want to increase the ProhibitSendQuota by 50MB for all users that have current ProhibitSendQuota greater than or equal to 100MB. This could potentially be a large task to do manually (checking every user and upgrading them), however it’s quick and easy using Exchange Shell.

Note that I used the $1st shell variable to hold the intermediate results so I could check the list of mailboxes obtained using the Where-Object cmdlet before actually running the foreach loop that bumps the send quota limits.
Saving the list in a variable also lets us check things again at the end using $1stGet-Mailbox to make sure everything worked as expected.
Exchange Shell opens up many new possibilities for managing Exchange Server and simplifies the administration of large number of users.
See the Using the Exchange Management Shell in TechNet for an introduction and The Exchange 2007 Wiki for more tips and examples.

ActiveSync is a technology that allows your mobile device to synchronize with the Exchange server for emails, calendars, contacts and task items in your mailbox. Exchange 2003 Service Pack 2 introduced a new feature called Direct Push which enables Exchange data in your mailbox to be sent in near real time. A long standing HTTPS request is maintained between the device and the Exchange server. When new items arrive to the Exchange mailbox, those changes are synchronized to the device. This enables ActiveSync to provide a similar mobile messaging experience as BlackBerry.

Exchange 2007 takes mobile messaging a step further by providing several improvements to ActiveSync features. Some of the key improvements include:

Mailbox Search – You are now able to search items in your entire mailbox without downloading the items to the mobile device.

Self Service Device Management – Exchange 2007 allows you to wipe data on a lost or stolen mobile device using a tool in the option interface in Outlook Web Access (OWA).

Handheld Lockup – A security policy can be set up to require a password be entered on your mobile device after a period of inactivity.

Out of Office Support – You are now able to configure an Out of Office Message directly from your mobile device.

HTML/Flagged Message – The HTML/flagged message can now be displayed properly on the mobile device.

In order to take full advantage of these new features, your mobile device will need the latest Windows Mobile 6.0 operation system. Ultimately, the mobile device’s operation system will largely determine the new features it supports.
For a full list of new features and supported devices, please refer to http://msexchangeteam.com/archive/2007/04/06/437572.aspx.

Exchange 2007 provides several improvements to the Out-of-Office (OOF) feature, which are accessible to users via Outlook 2007 or OWA. (If you are wondering why it is called OOF and not OOO, go here: http://msexchangeteam.com/archive/2004/07/12/180899.aspx)

Some of the improvements include:
1) Scheduled OOF Messages
You can now schedule when your OOF message is sent. One obvious benefit of this is that you can pre-create your OOF message(s) and set the desired schedule. When you leave the office, your messages are automatically sent, and upon your return, your OOF is automatically disabled. This is especially helpful when preparing for an extended period of absence, such as a business trip or vacation.

2) Improved Security
OOF messages will not be sent out in response to server-detected junk e-mail or internet mailing lists.

3) Improved Editing Controls
OOF messages can now be composed in HTML format and customized with all the same controls currently available for editing emails.

For more details on this feature, please see the Exchange 2007 OOF blog: http://msexchangeteam.com/archive/2006/10/06/429115.aspx.

Is OOF assistant one of your favourite features in Exchange 2007? Vote in our poll!

Where did those Public Folders go?! The answer is they are still there; just not as visible as they used to be. Exchange 2007 has been designed in such a way that Public Folders, although still available, require command-line like administration – a little bit clunky. But why use dated technology to store your files, emails, contacts and calendar items? The reported reason behind the slow demise of Public Folders is so that newer more efficient and functional technologies can prevail.

Enter SharePoint either in WSS 3.0(Windows SharePoint Services) or MOSS (Microsoft Office SharePoint Server, formerly SharePoint Portal Server) 2007 flavor. SharePoint provides an easily extensible platform for the sharing of information (something Public Folders didn’t offer). It allows administrators and users to create themed ‘sites’ and ‘lists’ that contain the same information, but on a rich platform that allows for version control (not available on PFs), check-in/check-out document control (not available on PFs), customizable views (not that easy to do on PFs) that can include multiple ‘webparts’ to display relevant information to viewers like work group calendars, link lists, key contacts etc. Other functionality like the quick creation of blog and wiki sites and configurable event driven email alerts further extend capabilities that were never even conceived in public folders.

Integration through Exchange 2007 comes at the client level (spare email routing requirements). SharePoint lists and objects can be added directly to the Outlook 2007 for direct access via the folder tree. OWA also integrates with SharePoint through the ‘Document Access’ feature (on a read-only basis) which allows remote users to open up local documents either via SharePoint URLs or UNC File Shares. And of course, if you want to provide direct access to your SharePoint Site collections, SharePoint can be placed on public facing systems and accessed via standard web browser and can even allow anonymous access to specific sections (no one has an excuse to NOT have a blog space now!)

So I say, out with the old (Public Folders) and in with the new (easily extensible rich featured SharePoint)!

For more information on SharePoint:

Microsoft Office SharePoint Official Site
http://office.microsoft.com/en-us/sharepointserver/FX100492001033.aspx

Microsoft Feature-by-Feature WSS 3.0 and MOSS 2007 Comparison http://download.microsoft.com/download/1/d/c/1dc632e8-71e1-466f-8a2f-c940f1438e0a/SharePointProductsComparison.xls

A great collaboration of SharePoint professionals and enthusiasts
http://www.sharepointblogs.com

A quick description of SharePoint from our Favorite Wiki Resource
http://en.wikipedia.org/wiki/SharePoint

One of our visitors recently asked whether there are any challenges with implementing Auto Discover effectively, and Owen, our resident expert, had this to say:

The simple answer, the setup can be as simple or as complex as your corresponding Exchange environment. In a simple Exchange 2007 scenario, where Exchange routes for only one domain, the setup is fairly straightforward. As part of the Exchange 2007 CAS server installation, the necessary Autodiscover components are installed by default on the CAS server, and corresponding SCP records created in Active Directory. For users connecting from remote systems, such as laptops or home computers, an external DNS record is required which directs requests to autodiscover.yourdomain.com to the external address of the Exchange 2007 CAS server.

For more complex Exchange 2007 implementations, with multiple email domains, many external users, and custom SSL certificates, there are a few issues to overcome. Microsoft recommends that a separate website be created for a heavily utilized Autodiscover service. For multiple email domains, individual DNS records for each email domain need to be directed to one common HTTP based URL, which should to be configured to redirect all requests to an HTTPS enabled CAS server hosting the Autodiscover web site. This allows the use of multiple email domains, but one common Autodiscover site and corresponding SSL certificate. Otherwise, individual Autodiscover websites would need to be created on the CAS server, each with individual SSL certificates, which would become cumbersome and expensive to manage.

As Exchange 2007 has introduced a new web based delivery mechanism for Offline Address Books (OABs) for Outlook 2007 clients, replacing the Public Folder method used for Outlook 2000 through 2003 clients, a fully functioning Autodiscover service is required to maintain full functionality for Outlook 2007 clients. The same is true for Unified Messaging and Availability services, therefore a good understanding of and well designed Autodiscover implementation is key to a successful Exchange 2007 deployment.

For further information, Microsoft has published a good description on their TechNet site that describes the various scenarios, and potential solutions.

Exchange administrators and end users alike will praise the new Exchange 2007 feature known as Autodiscover. This new feature, combined with Outlook 2007, makes the setup of new Outlook profiles as simple as logging into your webmail. Simply provide Outlook your email address and password, and your Outlook profile is configured with no other information required. Gone are the days of needing to know your server name, or hunting for the settings and location to input your RPC over HTTP information.

This feature is elegantly accomplished through external DNS records, and features included on the new Exchange 2007 Client Access Servers (CAS). Administrators simply need to publish a DNS record on the internet for autodiscover.yourdomain.com, directing it to the CAS server of their Exchange email environment. The CAS server then provides the settings to Outlook 2007, including their Exchange server name, their RPC over HTTP settings (now called Outlook Anywhere), their offline address book web URL, their display name, and any other information required. This information is provided through a file called Autodiscover.xml, which is hosted by an IIS website on the CAS server, secured through NTLM authentication and SSL communications.

Similar principles are in place for Outlook 2007 with regards to personal accounts provided through user’s Internet Service Providers or free email providers. Many providers have the required information published that allows a user to simply input their display name, email address, and password, and Outlook 2007 automatically applies the required settings. You no longer need information regarding your incoming mail server, outgoing mail server, authentication settings, etc.

Simply put, Autodiscover makes accessing your email that much easier.

Exchange can be considered to be a special purpose database application. The database is all the email and calendar entries in everybody’s mailbox. Rather than writing a database from scratch, Microsoft used the Jet Database engine also used by Microsoft Access. What’s that you say, gasping in horror, Microsoft Access is running my corporate email system??? Yup. It also runs your Active Directory, by the way.

Why doesn’t Microsoft switch to a “real” database like SQL Server. There’s certainly been talk about this for years, and a lot of rumors that would happen in E2K7, but it didn’t go down that way. And here’s the rub: Jet is actually way faster than SQL Server.

SQL Server is built from the ground up to be a true multi-client, multi-machine, client-server database, and that carries lots of overhead.

All the clients need to create ASCII “SELECT” statements, send them over a skinny little pipe to the server, have the server parse and execute the queries, and then send the results back over the same skinny pipe to be parsed on the client end and then served up to the application.

By contrast, the Jet data and Jet engine sits right on the same computer as the application accessing the db, and uses big fat shared memory communications. Meanwhile, Exchange has customized and optimized the heck out of it to ensure the database performance is super-optimized for Exchange. Because the Exchange store.exe process manages all the communications to the database, there is no need for all the overhead of a true client-server database.

For a true client-server app, there is no substitute for a nice robust relational database like SQL server. But for an app like Exchange where all communications to a given computer are already funneled through a “front-door” process (store.exe), it’s really not necessary… honest…

The UK’s ITWeek asks whether or not Outlook should be consigned to the scrap heap and replaced with Google mail.

This writer makes some interesting points, but we’ll wager he hasn’t tried Exchange 2007 yet. His primary issues – the fit between Outlook and Exchange 2007 is on our top eleven list, as are the improved Outlook Web Access, and how 2007 addresses expanding mailboxes with a revised architecture – are all addressed in Exchange 2007. As for Google – we’d like to hear from readers who are considering or using Gmail for their businesses – would you go with a service like Gmail for corporate mail? Why or why not?

In the last post I told you that E2K7 (that’s short for Exchange 2007) uses lots and lots of caching to improve performance, especially on 64-bit architectures where lots and lots of memory is available.

One way to improve performance is to keep all data in zippy-fast RAM (random access memory) as opposed to on slug-like hard drives. Trouble with RAM is that when you reboot the machine, all contents are wiped away whereas hard drives can keep everything between reboots. So you can’t only use RAM. The other problem with RAM is that there is never enough of it. 16GB is the practical limit of RAM you can load into a reasonably priced machine nowadays, whereas that same machine could take several Terabytes of disk.

So a compromise is needed. That compromise is caching. Caching means “fronting” the slow disk with fast RAM, and it relies on a property of most applications called “locality of access”. The actual number of disk pages accessed during any small period of time is much smaller than the total amount of disk, especially with Exchange. Think about the 1GB of data in your mailbox – how much of it do you ever use at the same time? That means that the whole “working set” of disk pages can be mirrored in RAM, if you have enough of it.

On the first read operation, Exchange can bring the 8 KB disk page containing your data (and some nearby data) into RAM. All subsequent reads and writes can occur onto the mirrored page in RAM. When the user moves onto something else 10 or 12 seconds later (an eternity to a CPU running with a 3GHz clock speed) the RAM-mirrored page begins “ageing”, and if it is not touched in a while, it is written back to disk so that the RAM can be used to mirror a different disk page that is in more active use.

This technique vastly reduces the number of iops to disk in favor of mops to RAM, which, are much, much faster, and don’t require that you pay for all those expensive disk spindles to supply you with your necessary dose of iops.

One caveat with Exchange is that it is a database (albeit a cheesy one, but more on that in another post), and so there is a requirement to write data back to disk. In databases, this is handled by writing changes to the database to a “transaction log” straight to disk without any pesky caching getting in the way. That way, if the server suddenly crashes, the combination of whatever is on the disk plus the transaction log can bring the database right back up to date. So in the case of Exchange, big caches don’t save on log write operations, but they sure save on store iops, which are by far the most common operation.

There are other ways of speeding up access to data other than caching, but they all require reorganizing the data radically and moving away from a more standard relational database organization to a super-customized disk organization, specifically geared towards your application. Even then, customized application-driven caching can still dramatically improve matters from there.

Taking a sub-optimal storage layout and throwing a 64-bit address space worth of caching at the problem as was done for E2K7 may not be elegant, but it sure gets the job done