The premise of the blog is that I spend a lot of time talking to customers about these concepts and it is clear that there is a lack of understanding around the concepts.  This blog (a two part series) is an explaination and introduction to the core concepts that customers will need to address in their own implementations.  Have a look here!

I’ll be presenting a search topic or two (duh!) and am really looking forward to meeting speakers and attendees in my new region!

For more info go to  http://spsevents.org/city/siliconvalley/SPSSV2013/Pages/Home.aspx

Not a ton of content there yet, but keep checking back for listings on the speaker lineup and session details.

Hope to see you there!

The event is taking place Saturday May 18th, 2013 at the Santa Monica College in beautiful Santa Monica.

Specific sessions haven’t yet been announced, but so far the speaker lineup is looking great.  A lot of familiar names and faces like Eric Overfield, Ivan Sanders, Naomi Moneypenny, Randy Williams and a whole bunch of others that I’m looking forward to meeting.

I’ll post more info about my sessions soon, but in the meantime find more info here:  http://spsevents.org/city/la/SPSLA2013/Pages/Home.aspx

Because I’m pompous, I chose to use 50 of history’s famous authors as the sample users    True, it doesn’t make for a “realistic” experience, but it tickles me to see Shakespeare and Steven King collaborate on documents together.

The package I created can be found here on CodePlex.

In the package you’ll find:

• A CSV file with all the test users
• A PowerShell script that iterates over the users in the CSV and adds them to AD
• A folder containing all the photos of the users
• A PowerShell script mapping the user photos to the new sample users

To add the users:

• Change variables in the AddUsersToAD.ps1 script appropriate to your environment and run on the domain controller to add users to the domain
• Configure and run your UPS Sync as you normally would
• Drag the photos in the UserPhotos folder into the User Photos folder (typically http://[host name]/User%20Photos/)
• Change variables in the Map_User_Photos.ps1 script appropriate to your environment and run on a SharePoint Server
• You should now have 50 new users in SharePoint with photos

Obviously I could have automated this more and even added  more options but this was enough for me.  I can add users to any environment within 15 minutes or so.

Hope it proves helpful for some of you!

See screenshot below…

I’ve decided to set up a local Hyper-V environment hosted on my laptop running Windows 8.  Let’s start with the specs.

Hardware

I have a Lenovo W530 series laptop with one standard issue spindle hard-drive and a second solid state hard drive (SSD) each 500GB.  I’ve reserved the SSD completely for running my virtual machines to isolate the host from the VMs from a disk perspective.

I’ve got 32 GB of RAM and a quad core Intel i7 2.60 GHz processors with 8 logical cores.

Software

I’m running Windows 8 Enterprise 64-bit with the Hyper-V role enabled.

My Setup

I wanted to set up an environment that could cover as many different needs and scenarios as possible.  Here are some of the requirements I set out to cover:

• Run both SharePoint 2010 and 2013 farms
• Set up multiple domains so that I could test and experiment with cross-domain challenges and issues
• Run both single-server farms as well as multi-server topologies

Here’s how I went about it:

For my single server 2013 farm, I used the “SharePoint 2013 Setup Guide for Developers for SharePoint 2013 RTM” created by Critical Path Training.   I definitely recommend it as it walks you through from setting up your virtual NICs and configuring your OS to configuring your environment for APP development and host header site collections.  If you follow this guide you will create a domain called wingtip.com and that’s one of my two domains.

As I mentioned, I wanted multiple domains and so my next step was to set up a second domain which I called it olenick.com.  My plan is to basically use the olenick.com domain for everything except the single-server 2013 farm which is covered by my wingtip environment.  In the olenick.com domain I have a domain controller (DC), a dedicated SQL Server and then any number of SharePoint farms that are attached to my domain and shared SQL server.

To make life easier for myself, as I was setting up the first environment using Critical Path’s document, I saved my virtual hard drive (VHD) to use as a base image.  That way as I’m spinning up new servers, I have a base Windows Server hard drive with all the updates installed which saves a TON of time -  installing updates takes FOREVER!  I also create base images for other purposes.  For example I have one with all SharePoint prereqs and bits installed, but the configuration wizard is not run yet.  I also have base images with visual studio and Microsoft Office installed.

So ultimately here is what my environment looks like.  This allows me to play with SharePoint 2010 and 2013.  Explore working with multiple domains for search crawling and other scenarios.  And I also have an Office 365 environment which I will use with my local dev environment to experiment with hybrid scenarios.

I don’t necessarily run this all at once, but I can cover a lot of bases with this!

Lastly, in the past I’ve used Microsoft’s IW Demo environment out of convenience because it comes with all the software  installed, plus content and users!  But, I’ve been burned enough times now with this quirky environment that I’m going to build my own from now on.  And that means I need to create my own content.  I plan to create some “content packs” which are deployable packages that add content to an existing farm.  Stay tuned as I’ll post what I come up with and learn during that process!

http://blogs.msdn.com/b/mvpawardprogram/archive/2013/02/25/mvp-monday-ediscovery-in-sharepoint-2013.aspx

This is the first in a short series of blogs in which I will write about custom content processing with FAST Search Server 2010 for SharePoint (FS4SP) and SharePoint 2013 Preview.  I think it will be interesting to lay out a couple of use cases for content processing, provide something of a how-to for the scenario(s) with FS4SP and then try to refactor the solutions for SharePoint 2013 Preview to see how the solutions will differ, what has become easier, more challenging, etc.

Very briefly, content processing refers to the act of modifying content as it travels through the document processing pipeline with your own custom code (or that of a 3rd party solution).  Use cases for custom processing can be anything from simple re-formatting (think dates, names, multi-valued properties, etc.) or as powerful as enhancing search index items with data from other systems.

In FS4SP, the place to do this is in the pipeline extensibility stage of the document processing pipeline.  The diagram below shows the flow in and out of the FS4SP index and a detailed section describing the item processing pipeline including the extensibility stage.  When the item being crawled hits the extensibility stage, your custom code can be called and the item can be impacted by your code.  As you can see, the custom processing happens towards the end of the pipeline and before the item makes it into the index.

The following link is Microsoft’s Technet overview on custom item processing:  http://msdn.microsoft.com/en-us/library/ff795801.aspx

So onto the first use case.

Use Case

I worked with a customer recently who was crawling HTML files that have custom meta tags in the header.  They wanted to know how to get the metadata stored in those meta tags into the index so they could use it as a refiner or to display it in the search results.  Additionally, some of these metadata fields contain date information and they wanted to know how to wire that info to existing date properties that they are using such as “Last Modified Date”.  This is a great use case because we can solve two separate, but related problems in one shot.

Custom Meta Tag

We’ll handle a single custom meta property for this example called MyCustomDate.  This is what the file looked like and if you notice, the date format is as follows:  “Tue, 31 Jul 2012 07:42:28 -0400″

So the first thing we need to figure out is how to get the property into the index – turns out that’s the easy part.  Being the expert  researcher that I am, I figured this out by tweeting Mikael Svenson and he told me the answer which is that they end up in the crawled properties under the “Web” category.  After all that research I had to take a break and get a snack   More on Mikael later in this post.

So to test this out, first we need to get the document into the index.  Instead of kicking off a crawl which can be a bit time consuming, I performed a docpush for my sample file. Docpush.exe is an out of the box FS4SP utility that allows you to add or remove one document at a time into the index.  Performing a docpush sends the item through all of the stages just as if it had been found during a full crawl.  It’s perfect for testing and debugging.  More about docupsh here:  http://technet.microsoft.com/en-us/library/ee943508.aspx

Once the item has been processed and is in the index, the custom meta tags are automatically added as crawled properties in the “Web” category – just as Mikael told me.  To find them navigate to Central Admin, Manage Service Apps, FAST Query SSA, FAST Search Administration.

From there, choose Crawled property categories and then “Web”.  In there all of the meta tags can be found.

Date Formats

So that is some progress, but right off this looks like trouble.  I see that my custom date crawled property is of type “Text” instead of “DateTime”.  Just to check whether this was a problem I went ahead and mapped the mycustomdate(Text) crawled property to one managed property of type “DateTime” and one of type “Text”

After creating and mapping these managed properties I performed my docpush again and executed a search for the document.  The results of my search told the story (see below) – the text property was populated but the date property was not.  This indicated that FS4SP didn’t want to map my text crawled property to the DateTime managed property.

Quick side-note.  The results in the screenshot above are from an out of the box search center, however I replaced the normal XSLT of the core results web part with my own that looks like this:

This will display a raw XML representation of all of the properties that are passed into the core results web part (in the fetched properties section of the web part).  So in my case I added mycustomdate and mytesttext to the fetched properties.

So the task at hand now is to convert the text string from MyCustomDate to something that FS4SP will recognize as a DateTime – and this is where the custom processing comes in.

Custom Pipeline Stage

Now typically when I create a custom processing stage I use Mikael Svenson’s example in this blog post as a starting point.  For those that don’t know Mikael, he has a tremendous amount of experience and knowledge around FS4SP, search and SharePoint in general and he is extremely generous with his knowledge.  I would also suggest that anyone working with FS4SP read his book Working with FAST Search Server 2010 for SharePoint.  He’s one of my SharePoint heroes

So, based on Mikael’s post, I created my custom processing console application and wired it up as he describes.

In the program, first I find the property and then get the date string which, if you remember from our document, is “Tue, 31 Jul 2012 07:42:28 -0400″

Then I reformat it with two lines of simple code.  In the example below “res” is a variable that contains the date.

DateTime MyDateTime = DateTime.Parse(res.First());

string myFormatedDate = MyDateTime.ToString(“s”) + “Z”;

Which yields this:  “2012-07-31T07:42:28Z”.  This is the suggested date format for FS4SP.  So it still remains a string at this point but we have it in the correct format.

Finally, I write that new value to a crawled property with the same name but give it the variant type “64″, which is DateTime.

Here is a reference for the different variant types.  I usually just peak at an existing property in Central Administration myself, but it is a good reference.  http://technet.microsoft.com/en-us/library/ff191231.aspx

Now, once I deployed the program, reset my docprocs (“psctrl reset” in the FS4SP Admin Console) and perform another the docpush again I saw the additional crawled property of type “DateTime”

It’s probably a good time to high-five someone at this point – we’re almost there.

Lastly, I mapped the new crawled property to a managed property called MyCustomDate in Central Administration and as you can see below, the custom date is now populated.

Now, as you can see, that the format is not particularly friendly to humans.  But don’t worry.  When I added the same crawled property to the “Write” (which is last modified) field, the date format came out perfectly like the screenshot below.  This is because there are XSLT templates used in the out of the box core search results web part handle formatting dates in the search results.

Conclusions

• Meta tags (even custom tags) are automatically surfaced as crawled properties and are found in the “Web” category.
• If you want to map a custom date field, make sure it is a crawled property of variant type “64″ (DateTime).
• Creating a custom pipeline stage to format a date is a fairly straight forward – especially if you start with some example code (thanks Mikael)

I like to say this a lot in my live presentations.  Don’t be afraid of writing a little code – even if you’re not a “developer”.  For encapsulated tasks like this (rather than large complex programs with lots of moving parts) you don’t have to be a rock star coder.  You can easily do this in an afternoon even if it’s your first time (assuming you have some familiarity with Visual Studio).  Just make sure you have one of your proper developer friends have a good look at it and it’s been tested thoroughly before it moves to production   I’m being funny here, but do make sure you test thoroughly and have experienced developers and testers take a look at the code before deploying to production!  But that said, don’t be afraid to roll up your sleeves and tinker around.  I’m not a “developer” but I don’t let this stuff scare me

Source Code

The environment I used to perform the tasks above was one of CloudShare’s pre-configured FS4SP servers.  I’ve made it available as a “permalink” here:  https://use.cloudshare.com/Pro/ShareEnv/0KZTTZGWVLUW

If you would like the source code click the “permalink”, create a CloudShare profile and once you get into the environment you’ll find the code on the desktop.

Stay Tuned

Stay tuned for the next blog in the series where we’ll explore a different use case.  Then in a subsequent post or two I’ll attempt to accomplish the same modifications in SharePoint 2013 so we can compare and contrast the tools and methods.

Happy processing!

The location itself was great – Microsoft provided their state of the art offices right in downtown Chevy Chase, MD.

Because of my schedule I was only able to attend two of the sessions but they were both great!

First, I saw Danny Jessee’s SharePoint 2010, Claims-Based Identity, Facebook, and the Cloud

This was a great session.  Jessee is an excellent speaker – extremely clear, prepared and knows the material inside and out so he could easily answer any questions that came up.  He started by giving one of the better explanations of claims-based authentication that I’ve seen and then went on to walk through and demo so really impressive integration between SharePoint 2010 and Facebook.  During the session he both posted live pictures, status updates and video live from within SharePoint.  Very cool stuff!

I also attended Rob Windor’s SharePoint 2010 Client-Side Development with the JSOM and jQuery .  As a speaker I love watching seasoned speakers like Rob so I can learn
and take notes.  He has great confidence, a polished but loose approach and serious command of the material.  This was sort of a 101 overview of how to incorporate client-side scripts into SharePoint 2010.  He gave great practical advise about how to choose an approach (since we have a few choices now) and a number of real-world examples.  It was standing room only and with good reason.  Killer session from a real expert!

Both of the sessions I delivered were related to Search.  First I talked about FAST Search Server 2010 for SharePoint troubleshooting in “Why is FAST so SLOW?!“.  This session is an overview of troubleshooting tools and techniques for FS4SP.  We had a great group of attendees who asked a lot of excellent questions.  I was a little worried this was going to be too dry, specific or technical.  But it drew a nice crowd and everyone was engaged the whole time.  Lots of fun!

My second session was the last of the day.  It was called “Simple Solutions for Creating Powerful Search Experiences” and provides a handful of lightweight solutions for the OTB SharePoint (or FS4SP) search centers.  The solutions focus on client-side scripts, CSS, XSLT and much of the discussion is designed to make non-developers less timid about attempting to make these kinds of customizations.

Metalogix hosted a fantastic SharePint where I had deep convos with UI/UX dev guru Tom Daly, AI wizard Naomi Moneypenny and seasoned IT pro Lynn Becker (@lwb2).

Overall, a really great day.  Thanks to the event sponsors and for the organizers for having me.  Look forward to coming back to DC again.  Keep an eye out as there are rumors of a Government focused event in the December timeframe!

Why is FAST so SLOW?!  which is about understanding and troubleshooting FAST Search Server 2010 for SharePoint.

The other is Best Practices: Simple Solutions for Creating Powerful Search Experiences in which I present several low-risk, high-impact solutions to improve the out-of-the-box search center in SharePoint 2010.

For more details about the conference, see http://www.sharepointsaturday.org/dc/default.aspx

These things are always a lot of fun and they’re completely free!!!!   Please come out and support the DC community!