For those of you who have tried SQL Azure Data Sync you likely know that the way that we execute synchronization from SQL Server to SQL Azure is through the use of SQL Agent.? One of the handy parts of SQL Agent is the ability to schedule synchronization based our your own requirements.? However, in some cases you actually want to execute synchronization based on some event that occurs within the database or even through an external application.? For that, there is a handy stored procedure called sp_start_job within the MSDB database which you can call to execute synchronization.? For example, I created a sync relationship between SQL Server and SQL Azure database using my AdventureWorks database and as a result there was a SQL Agent created called ‘SyncToSQLAzure-Sync_AdventureWorksLT2008′.? Using this, all I need to do is go to the msdb database (use msdb) and execute the following SQL:

exec sp_start_job ‘SyncToSQLAzure-Sync_AdventureWorksLT2008′

If you want to get information on the job, you can also use sp_help_job.

Liam

Author: Microsoft Sync Framework

Announcing Entity Framework & ADO.NET Data Services RTM!

We are excited to announce the RTM of the Visual Studio 2008 SP1 and the .NET Framework 3.5 SP1, including the RTM of the ADO.NET Entity Framework and ADO.NET Data Services, which raise the level of abstraction for database programming and supply both a new model-based paradigm and a rich, standards-based framework for creating data-oriented Web services. With this service pack, Visual Studio 2008 and the .NET Framework 3.5 also support SQL Server 2008, making the Microsoft platform the most comprehensive environment for database application development.

“The ASP.NET AJAX improvements and new capabilities like ADO.NET Entity Framework and ADO.NET Data Services meant we didn’t have to worry about any of the underlying plumbing and could simply focus on building a highly responsive and interactive experience for users,” says Galen Murdock, President and CEO at Veracity Solutions. 

For more information or to download check out http://msdn.microsoft.com/data  

Elisa Flasko
Program Manager, Data Programmability

Microsoft’s premier developer event, the Professional Developer Conference, is almost here. PDC 2008 will feature more than 160 sessions covering a wide range of topics including a number of great sessions from the Data Programmability Team, including the ADO.NET Entity Framework, ADO.NET Data Services and Project “Velocity”.

Developing Applications Using Data Services

Presenter: Mike Flasko

In the near future, applications will be developed using a combination of custom application code and online building block services, including data-centric services. In this session we discuss advancements in the Microsoft development platform and online service interfaces to enable seamless interaction with data services both on-premises (e.g., ADO.NET Data Services Framework over on-premises SQL Server) and in the cloud (e.g., SQL Server Data Services). Learn how you can leverage existing know-how related to LINQ (Language Integrated Query), data access APIs, data-binding, and more when building applications using online data.

Offline-Enabled Data Services and Desktop Applications

Presenter: Pablo Castro

The ADO.NET Data Services Framework (a.k.a. Project “Astoria”) introduced a way of creating and consuming flexible, data-centric REST services. By combining data services with the Microsoft Sync Framework, learn how to create offline-capable applications that have a local replica of their data, how to synchronize that replica with an online data service when a network connection becomes available, and how replicas can be used with the ADO.NET Entity Framework. Also, hear us talk about our plans, see the tools that help client- and server-side setup, and discuss the runtime components and APIs.

Entity Framework Futures

Presenter: Tim Mallalieu

The next version of the Entity Framework adds scenarios in the areas of   model driven development, domain driven development, simplicity, and integration. See a preview of production and prototype code for the next version of the Entity Framework as well as a candid discussion with members of the development team.

Project “Velocity”: A First Look

Presenter: Murali Krishnaprasad

It is predicted that all large applications will use a distributed data cache as the initial tier for all data access. This session presents an overview of “Velocity,” Microsoft’s distributed in-memory cache, and shows how it works with IIS, ASP.NET, ADO.NET and SQL Server Data Services.

Register at http://microsoftpdc.com/Registration/

See you there!!

Elisa Flasko
Program Manager, Data Programmability

We have a couple more synchronization samples for you to take a look at.  These C# samples are based on Sync101 which is a sample template that allows us to show specific concepts relating to Sync Framework.  All of these samples have been updated to work with the latest v 1.0 release of Sync Framework SDK . I am particularly excited about the remote change application sample (RCA) as it has been a common request to understand how to integrate Sync Framework with technologies like WCF and enable a service to manage the server provider componets of the Sync Framework.   Each of the samples use the the SQL MetadataStore store for the metadata information for each replica database. This greatly simplifies the logic required to implement data synchronization between data stores. In these samples three in memory data stores are used to show how data collaboration can be accomplished and how the Sync Framework helps to minimize P2P complexities such as data conflicts / resolution and data deletions.

The following are the samples that were just added to the Sync Framework Code Gallery.

Sync101 with Metadata Store v1.0
This sample uses the Metadata store to show the basic concepts that are required when building a Sync Framework application and illustrates many of the core concepts such as synchronization providers and the synchronization orchestrator. 

Sync101 RCA with Metadata Store v1.0
Expanding on the previous sample, this samples outlines the “remote change application” (RCA) pattern used over WCF.

Sync101 with Change Units
Expanding on the first samples, this sample illustrates how a data store can use change units to partition data and send only data that has changed. This sample also illustrate explicit transactioning (batched instead of per-update).

Liam

We have just posted the following two new samples to the Sync Code Gallery today for Sync Services for ADO.NET v2: Download Only Sync This sample demonstrates two-tier download-only synchronization using Microsoft Sync Services for ADO.NET v2.0. The sample solution is composed of two projects: Client, which includes the UI form, data access code, and synchronization agent code for Sync Services; and SampleServerSyncProvider, which includes server synchronization provider code for Sync Services. For more information, see the code comments that are included in each file.

Sync Adapter Builder
This sample utility demonstrates how to create synchronization commands by using the SqlSyncAdapterBuilder in Microsoft Sync Services for ADO.NET v2. The sample solution is composed of one project that includes the UI form, data access code, and SqlSyncAdapterBuilder code. The SqlSyncAdapterBuilder is typically used at run time to generate synchronization commands, which are in the form of SQL queries. This provides an alternative to writing these commands manually at design time. In this sample, we use the SqlSyncAdapterBuilder to output synchronization commands. You can output commands that work with SQL Server change tracking (available in SQL Server 2008) or a custom change tracking system. Running this application enables you to see how the commands are structured, and to copy those commands for use in applicatons where you will specify commands manually.

Liam

Right on the heels of our launch of Microsoft Sync Framework v1, we have just posted our first CTP for Microsoft Sync Framework for Devices. As of right now, you can download this CTP for Windows Mobile from the Microsoft Download Center.  As we have shown in previous webcasts and conference demos, this is great for enabling synchronization from devices to data stores such as SQL Server Data Services or even corporate web services.  Perhaps even to write an offline SmugMug application for synchronizing your device photos with your SmugMug account.

Not to be confused with the previous CTP release of Sync Services for ADO.NET for Devices (a component of the Sync Framework) that is specifically meant for synchronizing an ADO.NET enabled server database with a SQL Server Compact device database, this CTP release of Sync Framework for Devices includes the Core Sync Framework runtime that gives the ability to exchange information with any data store or file system on the device, and not just SQL Server Compact databases.

This release contains only the core sync engine you would find in “synchronization.msi” in the PC release. Other features such as the metadata store and file sync provider are not included: some of the omitted features don’t necessarily make sense on devices, others will appear in a future CTP. We are looking for feedback on which are which. In particular, we want to know: what are your scenarios for using the different aspects of the Sync Framework on a device?

We also decided to produce the CTP for a limited platform set: Windows Mobile 5 and 6 for ARM processors. Supporting this subset allows us to give a CTP usable by a very large percentage of mobile developers while still limiting the test matrix for this first CTP. We expect to expand the supported devices and platforms for RTM and we are considering the scope of that as we go forward.

So what exactly is in this release?

Microsoft Sync Framework core runtime

• Managed (.NET Compact Framework) and unmanaged/native (ARM)
• Supported platforms are Windows Mobile 5 and 6 (ARM processors only in CTP1)

Installer with Visual Studio integration

• Support for adding a reference to the Sync Framework in a mobile project using Visual Studio 2008

As always, we look forward to your feedback on the Sync Framework Forum.

Liam

Today’s release of Microsoft Sync Framework includes the latest version of Sync Services for ADO.NET.  A great deal of work has gone into this release and I thought I would take some time to point out some of the top new features in version 2 since our previous release.

• Peer-to-peer synchronization support:

Beyone the traditional hub-and-spoke synchronization support built into Sync Services for ADO.NET.  v2 now supports peer-to-peer synchronization.  The focus with this part of the API is collaborative applications. For example, in an application that allows users to share project notes, project team members often require a local copy of data that they can work with. When they have made changes, they can synchronize with another team member to exchange changes. 

• Sync Services is now based on Microsoft Sync Framework:

Sync Services now uses Microsoft.Synchronization.dll. If you installed Sync Services by using SyncSetup.exe, this DLL is already installed. This enables Sync Services for ADO.NET to take advantage of all the features and capabilities built into the Sync Framework.

• Synchronize with devices: 

Sync Services for ADO.NET 1.0 enabled synchronization between a server database and a SQL Server Compact 3.5 database on the desktop. Now, you can also synchronize between a server database and a SQL Server Compact 3.5 database on a device.

• Support for SQL Server 2008 Integrated Change Tracking:

If you are using SQL Server 2008, Sync Services for ADO.NET has the ability to take advantage of the new SQL Server integrated change tracking feature. This feature addresses many of the issues of custom-tracking systems and provides a straightforward way to track changes. Change tracking is also supported by the Local Database Cache in Visual Studio 2008 SP1. Developers can now specify that the Configure Data Synchronization wizard should enable SQL Server change tracking on the server and generate the commands necessary to select and apply changes to the server database. Unlike custom change tracking systems, SQL Server change tracking does not require any schema changes in the server database.

• Support for the new data types in SQL Server 2008:

Sync Services for ADO.NET has added support for a number of new data types including those added in SQL Server 2008.

• The ability to trace the synchronization process: 

In distributed applications, tracing can be crucial because it enables you to troubleshoot issues that might otherwise be difficult to identify. Sync Services includes tracing for the client and server synchronization providers.

Liam Cavanagh

I am happy to announce the general availability of Sync Framework v1 and Sync Services for ADO.NET v2.  As of today you can download the Sync Framework SDK in 11 languages including Chinese (Hong Kong), Chinese (Simplified), English, German, French, Italian, Japanese, Korean, Portuguese, Russian and Spanish for AMD64, IA64 and x86 processors from the Sync Framework Download Center.

What is Sync Framework?

Imagine being able to build a solution that seamlessly exchanges contact information between Outlook, a database contact management application, your mobile device and your service based contact management system.  Or how about a mobile device that connects with other devices to exchange pictures and videos.  How about being able to take data from any of your enterprise databases, file or enterprise systems and make it available offline for users to modify and sync back up to the enterprise.  All of these capabilities are possible with the Sync Framework and best of all, it is free on Windows platforms and licensable on non-Windows platforms!

Just a few examples of companies that are already using the Sync Framework include:

• SmugMugwho has added support for the Sync Framework to enable developers to build rich offline applications that can allow photos to be easily shared on friends and families local computers.
• InterScape have embedded Sync Framework into their Customer Relationship & Management (CRM) solution to enable sales people to synchronize enterprise files and data for offline access.
• Fujitsu Siemenswill embed the SyncToy file synchronization solution (powered by Sync Framework) on their STORAGEBIRD external drives to enable seamless file synchronization between devices and computers.

Sync Framework Highlights

Some of the highlights of this release include:

• Sync support to new and existing applications, services, and devices
• Collaboration and offline capabilities for any application
• Roam and share information from any data store, over any protocol, and over any network configuration
• Leverage sync capabilities exposed in Microsoft technologies to create sync ecosystems
• Extend the architecture to support custom data types including files

Sync Services for ADO.NET v2 Highlights

Sync Services for ADO.NET is a Microsoft Sync Framework powered solution for synchronizing ADO.NET enabled databases in offline and collaboration scenarios. Sync Services for ADO.NET allows developers who are familiar with the concepts of ADO.NET to apply that knowledge to data synchronization through a very similar set of APIs to that of ADO.NET. Sync Services for ADO.NET provides the flexibility of a programming model like offline datasets and a richer synchronization feature set like that found in Merge replication. Sync Services for ADO.NET also supports synchronization over services, such as Windows Communication Foundation (WCF).

Some of the Sync Services for ADO.NET highlights include:

• Offline Database Synchronization: With Sync Services for ADO.NET you are able to build a solution where multiple remote clients connect and synchronize to a central ADO.NET database in a Hub-and-Spoke configuration. This enables occasionally connected devices to periodically connect and synchronize changes with a central ADO.NET database server. This topology is a common solution for remote workers such as sales reps or field service workers.
• Collaboration Between Databases: Sync Services for ADO.NET also includes Peer-to-Peer capabilities. Through a custom Peer provider, collaboration between two or more SQL Server databases can occur (support is not available for SQL Server Compact). Unlike a Hub-and-Spoke architecture, this provider enables a SQL Server database to communicate and exchange information with any other SQL Server database. This type of scenario is useful in group scenarios where users (such as auditors) need to update information and then collaborate those changes with other group members.
• SQL Server 2008 Integrated Change Tracking: If you are using SQL Server 2008, we recommend that you use the SQL Server change tracking feature. This feature addresses many of the issues of custom-tracking systems and provides a straightforward way to track changes. Change tracking is also supported by the Local Database Cache in Visual Studio 2008 SP1. Developers can now specify that the Configure Data Synchronization wizard should enable SQL Server change tracking on the server and generate the commands necessary to select and apply changes to the server database. Unlike custom change tracking systems, SQL Server change tracking does not require any schema changes in the server database. For more information, see the Visual Studio 2008 documentation.

Sync Framework Pricing

Sync Framework will be licensed free on Windows platforms.  In addition, we are also licensing the specifications and a source code porting kit to developers who want to implement Microsoft Sync Framework solutions on non-Windows platforms.

To learn more, visit http://msdn.microsoft.com/sync

I also want to send a special thank-you to those of you who have provided feedback throughout our CTP’s and have helped make this a great release!

Liam Cavanagh

Like the picture? It’s not really relevant to anything contained in the following post except that it is an excellent visual aid for describing my professional life of late. “To leave whatever you’re doing, you have to change your course.” Yep, that about sums it up.

Let it never be said that I can’t be motivated to do things. Roughly a month and a half has passed since my last post and at last the combination of guilt over my silence and my overwhelming need to be heard has driven me into action. I’m leaving out the indirect financial motivation (job-based compensation) for me to stay on top of this but it also comes into play as well. All of it adds up to me being prodded into action.

I did want to revisit my experience at the 2008 MySQL Conference & Expo. My attendance could be summarized this way: I attended, I spoke, I met, I listened, I left. Unfortunately taking this approach would leave out a lot of details that breathe a little more life into my retelling of the experience.

First off, let me just say that despite a technical hitch that delayed the start of my presentation by 10 minutes, I managed to keep an audience of around 20 people around long enough to sit through the entire session. This is a personal victory and a milestone. The last time I spoke at a conference, it was to a rapt audience of 3 people – 1 fast asleep, 1 who was sitting down to rest his weary legs (before he too, fell asleep, presumably) , and 1 person from my company who was essentially, paid to be there. Yes, I’ve come a long way, and while I was not offered any barleywine by any of the attendees (*tsk tsk*), I appreciated the attentiveness and polite questions afterward.

If you missed my presentation and for some crazy reason would like to see my slide deck, have at it, and don’t forget to let me know what you think or share any comments or questions that you might have: MySQL%202008%20SOA%20Data%20Access%20Presentation.ppt

I was an attendee at a couple of sessions at the conference myself – the most interesting one for me was MySQL, LINQ and the ADO.NET Entity Framework, a presentation by Dave Sceppa, who is a Program Manager at Microsoft (I came late, so I just caught the tail end of Dave’s intro where he explained that Reggie Burnett couldn’t make it due to an injury he suffered during the trip? I’m not sure what happened). Dave had a lot of interesting things to present about ADO.NET and LINQ in particular. I had an opportunity to meet and introduce myself him after his presentation – very nice guy and one that I would recommend anyone who is interested in ADO.NET seek out at trade shows.

I was also at the MySQL Partners meeting where it was announced that MySQL was considering rolling out some new MySQL features (specifically ones related to backup and encryption) into MySQL Enterprise only. Jeremy Cole, whose blog more or less broke this news, was at the meeting and was pointed in his requests to the MySQL executive team for clarification and confirmation of the facts around this decision. Based on the rash of comments on his post and the prolific blogosphere hand-wringing over this announcement, it was clear that a lot of people in the MySQL community weren’t happy about it. In fact, some claimed that Sun, who had just completed their acquisition of MySQL was now starting to close source it. In the software world, close sourcing an open source product is seen as the equivalent of Darth Vader freezing Han Solo in carbonite. While Han (MySQL) is technically still alive after his imprisonment, his state is considered unchangeable and subject to the whims of the Empire (Sun). It should be noted that after all of the fuss over this consideration, Sun seems to have backtracked on this a bit, but there’s still debate over Sun’s intentions with regards to making dividends on their billion-dollar investment in MySQL.

I was not as shocked and dismayed by this announcement / consideration as others apparently were. To me, it comes down to simple economics: why wouldn’t Sun be looking for ways to add value to MySQL Enterprise and thus help justify why customers should pay for it versus simply downloading and using a free edition? Of course, some may argue that as an employee of a software company that makes money on closed source software, my perspective is easily dismissed. Without wanting to explain DataDirect’s business model in great detail, I’ll just say that if our products did not demonstrate value over our competition (which is almost always free or ridiculously cheap), we would have gone out of business a long time ago. So yeah, I guess I’m not afraid to say that because value is what differentiates software companies that make money on their products and services (and stay in business for over 20 years) and the ones that don’t make money (and consequently, fail to stay in business).

My final thought on my trip is a bit lighthearted – any of you who consider yourselves beer snobs (a la, yours truly) and find yourself hunting for a place to buy and sip great beer in Bay Area would do well to consider showing up at the City Beer Store in San Francisco. The owners were terrific – extremely friendly and helpful, and the selection was terrific (I scored 2 bottles each of Russian River Beatification (Batch 002) and Supplication!). I managed to stuff my suitcases full of enough bottles of deliciousness that the TSA saw fit to inspect my bags, though I am happy to report that they did so professionally and did not “borrow” any of my precious goodies for themselves. (THANK YOU!!)

I’m planning to move up to a more regular update schedule, so look for a new entry well within the month and a half long interval that I established previously. Oh, and look for things to get alternately more business-focused AND more whimsical. You were warned. ^_^

I thought I would take the chance to talk to you a little bit about a major enhancements we have made with synchronization in SQL Server 2008. The feature I am referring to is called SQL Server 2008 Integrated Change Tracking. The idea behind this feature is that it enables SQL Server to track data changes to your database rather than relying on you to create a change tracking technique.

Why is this feature so great? Well in my experience I have found that DBA’s are less then enthusiastic when you tell them that in order to track changes, they will need to add triggers to each of the tables. Then they will either need to add an additional tracking column to that table (or to a separately linked table) and will need to create one tombstone table per table to track deletes. Oh, and by the way these triggers will need to be fairly complicated because you really need to deal with all of the isolation issues that come along with proper change tracking… and cross your fingers that these changes do not affect your existing applications. Not a fan of that idea? I am not surprised.

Well with SQL Server 2008, you can now simply turn on an option to enable change tracking. Then you tell SQL Server which tables you want it to monitor. From that point SQL Server will start monitoring these changes and store them in a separate change tracking table which you have access to. If you want to get the changes since a certain point in time you simply join the ChangeTable to your base table and presto you have all of the inserts, updates and deletes.

As an added bonus, this feature has been integrated into the existing Visual Studio SP1 beta that is now available for download. Guy Burstein has a great summary of this feature and I am going to steal one of his screen shots below.

Liam Cavanagh