How to Extract Email Attachments By Pipeline

Introduction

In this article I will demonstrate step by step instructions of how to extract email attachments from POP3 adapter by using custom receive pipeline.

Problem Statement

You can have some customers that need to integrate with your integration middle-ware using POP3. So, They will send emails with multiple attachments and these attachments represent the messages that you need to process.

Solution

There are 2 BizTalk solutions that we can implement either in Message Level or Orchestration level. As performance perspective, a message level is much better because orchestration level has much persistence points which can reduce the performance of our integration solution.
If you would like to see the orchestration solution you can check this link.

To process email attachments through message level, then we need to process them using pipeline.
So, we need to implement custom receive pipeline and add it to disassemble part of receive pipeline
For now let's start our walk-though demonstration.

Create Custom Receive Pipeline

1. Create a new solution as shown in figure 1
   
   Figure 1- Creating a new Solution
   
   
2. Create a new class library projects as shown in figure 2
   
   Figure 2- Creating a new Class library project
   
   
3. Remove Class1.cs file
4. Create a new class as shown in figure 3
   
   Figure 3- Creating a new class
   
   
5. Add Microsoft.BizTalk.Pipeline.dll from this path %BTSINSTALLPATH%
6. Add Microsoft.BizTalk.Pipeline.Components.dll as a reference from this path %BTSINSTALLPATH%\Pipeline Components
7. Implement the IBaseComponent, IComponentUI interfaces as shown in figure 4
   
   namespace TechNetWiki.AttachmentsExtractor.PipelineComponent

   {

       using System;

       using System.IO;

       using System.Text;

       using System.Reflection;

       using System.ComponentModel;

       using Microsoft.BizTalk.Message.Interop;

       using Microsoft.BizTalk.Component.Interop;

       using System.Runtime.InteropServices;

     

       [ComponentCategory(CategoryTypes.CATID_PipelineComponent)]

       [ComponentCategory(CategoryTypes.CATID_DisassemblingParser)]

       [Guid("01F04E58-7AD8-42EF-A3CB-D939B960E8F9")]

       public class AttachmentsExtractor : IBaseComponent, IComponentUI

       {

           #region IBaseComponent Members

           /// <summary>

           /// Gets Description of the component

           /// </summary> 

           public string Description

           {

               get

               {

                   return "Email Attachments Extractor";

               }

           }

     

           /// <summary>

           /// Gets Name of the component

           /// </summary> 

           public string Name

           {

               get

               {

                   return "EmailAttachmentsExtractor";

               }

           }

     

           /// <summary>

           /// Gets Version of the component

           /// </summary>

           public string Version

           {

               get

               {

                   return "1.0";

               }

           }

     

           #endregion

     

           #region IComponentUI members

           /// <summary>

           /// Component icon to use in BizTalk Editor

           /// </summary>    

           public IntPtr Icon

           {

               get

               {

                   return System.IntPtr.Zero;

               }

           }

     

           /// <summary>

           /// The Validate method is called by the BizTalk Editor during the build

           /// of a BizTalk project.

           /// </summary>

           /// <param name="obj">An Object containing the configuration properties.</param>

           /// <returns>The IEnumerator enables the caller to enumerate through a collection of strings containing error messages. These error messages appear as compiler error messages. To report successful property validation, the method should return an empty enumerator.</returns>

           public System.Collections.IEnumerator Validate(object obj)

           {

               return null;

           }

           #endregion

       }

   }
   Figure 4-Implementing  IBaseComponent, IComponentUI
   
   
8. Implement IDisassemblerComponent interface which is mandatory to implement GetNext and Disassemble methods 
   • GetNext method returns messages resulting from the disassemble method execution 
   • Disassemble method used to split or process the incoming message document
     
9. When we receive an email we will receive it as a message associated with multi-parts messages so In Disassemble method we need to loop in each Part of the message and create a new message.We will will start looping from index 1 because index zero is the body content which we are not interested as shown in figure 5
   #region IDisassemblerComponent members

           /// <summary>

           /// Returns messages resulting from the disassemble method execution

           /// </summary>

           /// <param name="pc">the pipeline context</param>

           /// <returns></returns>

           public IBaseMessage

               GetNext(IPipelineContext pc)

           {

               // get the next message from the Queue and return it

               IBaseMessage msg = null;

               if ((_msgs.Count > 0))

               {

                   msg = ((IBaseMessage)(_msgs.Dequeue()));

               }

               return msg;

           }

           /// <summary>

           /// called by the messaging engine when a new message arrives

           /// </summary>

           /// <param name="pc">the pipeline context</param>

           /// <param name="inmsg">the actual message</param>

           public void Disassemble(Microsoft.BizTalk.Component.Interop.IPipelineContext pc, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)

           {

               var partName = string.Empty;

               // we start from index 1 because index zero contain the body of the message

               // which we are not interested

               for (int i = 1; i < inmsg.PartCount; i++)

               {

                   IBaseMessagePart currentPart = inmsg.GetPartByIndex(i, out partName);

                   Stream currentPartStream = currentPart.GetOriginalDataStream();

                   var ms = new MemoryStream();

                   IBaseMessage outMsg;

                   outMsg = pc.GetMessageFactory().CreateMessage();

                   outMsg.AddPart("Body", pc.GetMessageFactory().CreateMessagePart(), true);

                   outMsg.Context = inmsg.Context;

                   currentPartStream.CopyTo(ms);

                   string attachmentContent = Encoding.UTF8.GetString(ms.ToArray());

                   MemoryStream attachmentStream = new System.IO.MemoryStream(

                   System.Text.Encoding.UTF8.GetBytes(attachmentContent));

                   outMsg.GetPart("Body").Data = attachmentStream;

                   outMsg.Context.Write("ReceivedFileName", "http://schemas.microsoft.com/BizTalk/2003/file-properties", partName);

                   _msgs.Enqueue(outMsg);

               }

           }

           #endregion
   Figure 5- Implementing IDisassemblerComponent interface
   
   the Full code of custom pipeline as show in figure 6
   
   
   namespace TechNetWiki.AttachmentsExtractor.PipelineComponent

   {

       using System;

       using System.IO;

       using System.Text;

       using System.Reflection;

       using System.ComponentModel;

       using Microsoft.BizTalk.Message.Interop;

       using Microsoft.BizTalk.Component.Interop;

       using System.Runtime.InteropServices;

       [ComponentCategory(CategoryTypes.CATID_PipelineComponent)]

       [ComponentCategory(CategoryTypes.CATID_DisassemblingParser)]

       [Guid("01F04E58-7AD8-42EF-A3CB-D939B960E8F9")]

       public class AttachmentsExtractor : IBaseComponent, IComponentUI, IDisassemblerComponent

       {

           #region IBaseComponent Members

           /// <summary>

           /// Gets Description of the component

           /// </summary>  

           public string Description

           {

               get

               {

                   return "Email Attachments Extractor";

               }

           }

           /// <summary>

           /// Gets Name of the component

           /// </summary>  

           public string Name

           {

               get

               {

                   return "EmailAttachmentsExtractor";

               }

           }

           /// <summary>

           /// Gets Version of the component

           /// </summary>

           public string Version

           {

               get

               {

                   return "1.0";

               }

           }

           #endregion

           #region IComponentUI members

           /// <summary>

           /// Component icon to use in BizTalk Editor

           /// </summary>     

           public IntPtr Icon

           {

               get

               {

                   return System.IntPtr.Zero;

               }

           }

           /// <summary>

           /// The Validate method is called by the BizTalk Editor during the build

           /// of a BizTalk project.

           /// </summary>

           /// <param name="obj">An Object containing the configuration properties.</param>

           /// <returns>The IEnumerator enables the caller to enumerate through a collection of strings containing error messages. These error messages appear as compiler error messages. To report successful property validation, the method should return an empty enumerator.</returns>

           public System.Collections.IEnumerator Validate(object obj)

           {

               return null;

           }

           #endregion

           /// <summary>

           /// this variable will contain any message generated by the Disassemble method

           /// </summary>

           private System.Collections.Queue _msgs = new System.Collections.Queue();

           #region IDisassemblerComponent members

           /// <summary>

           /// Returns messages resulting from the disassemble method execution

           /// </summary>

           /// <param name="pc">the pipeline context</param>

           /// <returns></returns>

           public IBaseMessage

               GetNext(IPipelineContext pc)

           {

               // get the next message from the Queue and return it

               IBaseMessage msg = null;

               if ((_msgs.Count > 0))

               {

                   msg = ((IBaseMessage)(_msgs.Dequeue()));

               }

               return msg;

           }

           /// <summary>

           /// called by the messaging engine when a new message arrives

           /// </summary>

           /// <param name="pc">the pipeline context</param>

           /// <param name="inmsg">the actual message</param>

           public void Disassemble(Microsoft.BizTalk.Component.Interop.IPipelineContext pc, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)

           {

               var partName = string.Empty;

               // we start from index 1 because index zero contains the body of the message

               // which we are not interested

               for (int i = 1; i < inmsg.PartCount; i++)

               {

                   IBaseMessagePart currentPart = inmsg.GetPartByIndex(i, out partName);

                   Stream currentPartStream = currentPart.GetOriginalDataStream();

                   var ms = new MemoryStream();

                   IBaseMessage outMsg;

                   outMsg = pc.GetMessageFactory().CreateMessage();

                   outMsg.AddPart("Body", pc.GetMessageFactory().CreateMessagePart(), true);

                   outMsg.Context = inmsg.Context;

                   currentPartStream.CopyTo(ms);

                   string attachmentContent = Encoding.UTF8.GetString(ms.ToArray());

                   MemoryStream attachmentStream = new System.IO.MemoryStream(

                   System.Text.Encoding.UTF8.GetBytes(attachmentContent));

                   outMsg.GetPart("Body").Data = attachmentStream;

                   outMsg.Context.Write("ReceivedFileName", "http://schemas.microsoft.com/BizTalk/2003/file-properties", partName);

                   _msgs.Enqueue(outMsg);

               }

           }

           #endregion

       }

   }
   Figure 6- Custom pipeline full code
   
   
   
10. Build the project then copy the generated TechNetWiki.AttachmentsExtractor.PipelineComponent.dll to %BTSINSTALLPATH%\Pipeline Components

Create Receive Pipeline

1. Create a new BizTalk Project by right click solution and select new project then select Empty BizTalk Server Project and name it TechNetWiki.AttachmentsExtractor.Pipelines as shown in figure 7
   
   Figure 7- Create Pipeline BizTalk Project
   
   
2. Right click TechNetWiki.AttachmentsExtractor.Pipelines project -> Select Add-> New Item -> Select Receive Pipeline-> Name it AttachmentsExtractorPipeline as shown in figure 8 
   
   Figure 8- Create a new receive pipeline
   
   
3. Right click to ToolBox window ->select Choose Items... -> select BizTalk Pipeline Components -> check EmailAttachmentsExtractor as shown in figure 9
   
   Figure 9- Add pipeline component to ToolBox
   
   
4. Drag drop EmailAttachmentsExtractor to Disassemble part as shown in figure 10 
   
   Figure 10- Setting pipeline component in Disassemble part
   
   
5. Right click project TechNetWiki.AttachmentsExtractor.Pipelines -> select Properties -> select Sign tab-> check Sign the assembly -> choose a strong name key file -> select New and rename it to key.snk -> select Deployment tab -> set Application Name to TechNetWiki.AttachmentsExtractor
6. Right click TechNetWiki.AttachmentsExtractor.Pipelines -> select Deploy

Configure POP3 Adapter

1. Open BizTalk Server Administration Console 
2. Right click TechNetWiki.AttachmentsExtractor - Receive Port -> select New -> One-way Receive Port... -> change name to Rcv_EmailTest -> select Receive Location tab -> change name to RcvLoc_EmailTest
3. Select POP3 adapter-> click Configure button -> set POP3 adapter configure as shown in figure 11 In this demo I am using my hotmail account to configure POP3 adapter
   
   Figure 11 - Configuring POP3 adapter
   
   
4. Create a new send port and name it Snd_EmailTest -> select File Adapter and configure it as shown in figure 12
   figure 12- Creating send port
   
   
5. Select Filters tab then set filter to our receive port name as shown in figure 13
   
   Figure 13- Setting filter

Test the Solution

1. Send email with sample attachment message to the configured email that we configured on POP3 adapter which is my demo YouUserName@hotmail.com
2. Check send port location

Sample Code

You can find the sample source code in the following link

Conclusion

In BizTalk Server there are two ways to extract email attachments either using custom receive pipeline or orchestration

In this article, I walked-through in step by step how to create a custom receive pipeline to extract email attachments.

See Also

Read suggested related topics:

• Splitting POP3 Email Attachments using orchestration
• MSDN Pop3 Adapter documentation

Another important place to find a huge amount of BizTalk related articles is the TechNet Wiki itself. The best entry point is BizTalk Server Resources on the TechNet Wiki.

---

An Easy Way to Get BizTalk Alerts

Introduction

As a BizTalk administrator, you need to know about any issues in your BizTalk environment. It is crucial to know about these issues before they become real problems or cause any unnecessary outage for any integration services provided by your BizTalk server.

In this article I will demonstrate an easy way to send alerts in case there are any errors or warnings in the BizTalk environment.

Problem Statement

You may have some customers connecting to your integration middle-ware through receive adapters such as POP3, SFTP, FTP, File, etc. In some cases you may get some warnings and errors which affect these receive adapters and make them disabled once the error threshold is exceeded.

In many instances you would not be able to use the BizTalk Administration Console for alerting as these entries do not even register in the BizTalk Message Box.

Therefore, it is very important to be notified before the end users prompted with warning or error messages when they are trying to upload into integration middle-ware. It is equally important to take a proactive approach to resolve any upcoming issues you might be experiencing with your BizTalk server.

Solution

You might be attempted to use monitoring tools like SCOM or BizTalk360,..These tools are not free and some companies might find them costly to licence and implement.

If your project does not have a budget to deploy these motioning tools, there is an easy way to get BizTalk alerts by using a combination of PowerShell scripting and the built-in Windows Task Scheduler through the following steps:

1. Select Windows Start Button ->Control Panel->Administrative Tools -> Task Scheduler as shown in Figure 1 or use the short cut command in run window control schedtasks
   
   Figure 1 - Accessing Task Scheduler
2. In Task Scheduler window select Microsoft folder->click on create task -> in General Tab Name the task as shown in Figure 2
   
   Figure 2- Create Task
3. Select Trigger Tab->Click New button-> in the Begin the task select on an event ->select custom->click New Event  Filter button-> in Event Sources select BizTalk -> In case you want to filter by using event id you can set event id as shown in Figure 3 
   
   
   Figure 3- Create Trigger
   
   
4. Select Action Tab->Click New Button->In Action select Send an e-mail->set email settings properties as shown in Figure 4
   
   Figure 4 - Create Exchange Email Action
5. If you want to to add attachment of event logs to be included in email, then you need to create a new action to run the following command  which write your filter log into a text file in my case I filtered on EventId 5740 which is pop3 warning that happens once pop3 user not authenticated and save it in this path C:\SendEmail\temp\logs.txt
   del C:\SendEmail\temp\logs.txt

   wevtutil qe Application "/q:*[System [(EventID=5740)]]" /f:text /rd:true /c:1> C:\SendEmail\temp\logs.txt
6. Save command into cmd file and rename it into path C:\SendEmail\WriteLog.cmd
7. Create new action -> In Action field select start a program-> Set Program/Script Path as shown in Figure 5
   
   Figure 5 - Create Command Action
   
   
8. Sort action and make sure that running the command action in the first action as shown in Figure 6
   
     Figure 6- Sort actions
   
   
9. In case you do not want to use exchange mail server, then you can write the following powershell script and save it in this path C:\SendEmail\SendEmailScript.ps1
   
   $emailSmtpServer="smtp.gmail.com"

   $emailSmtpServerPort="587"

   $emailSmtpUser="yourusername@gmail.com"

   $emailSmtpPass="yourpassword"

   $emailFrom="fromusername@gmail.com"

   $emailTo="sendtousername@gmail.com"

   $emailMessage=New-Object System.Net.Mail.MailMessage( $emailFrom , $emailTo )

   $emailMessage.Subject="BizTalk Error"

   $emailMessage.IsBodyHtml=$true

   $emailMessage.Body=@"

   <p>There is a BizTalk issue</p>

   <p>Check the attachment</p>

   "@

   $emailAttachment=New-Object System.Net.Mail.Attachment("C:\SendEmail\temp\logs.txt")

   $emailMessage.Attachments.Add($emailAttachment)

   $SMTPClient=New-Object System.Net.Mail.SmtpClient( $emailSmtpServer , $emailSmtpServerPort )

   $SMTPClient.EnableSsl=$true

   $SMTPClient.Credentials=New-Object System.Net.NetworkCredential( $emailSmtpUser , $emailSmtpPass );

   $SMTPClient.Send( $emailMessage )
10. Create another command file to run powershell script as below and save it in this path C:\SendEmail\RunPowerShellScript.cmd
    
    powershell C:\SendEmail\SendEmailScript.ps1
    
    Now SendEmail folder should be as shown in Figure 7
    
    
    Figure 7- Files inside SendEmail Folders
    
    
11. Update Send Email Action which we created in step 4 to run a program with path C:\SendEmail\RunPowerShellScript.cmd as shown in 
    Figure 8- Edit Action to use PowerShell script
12. To test your work, create pop3 receive location and set a wrong username and password

Conclusion 

In this article, I walked-through in step by step how to use an easy way to send any BizTalk alerts to specific email using a built-in Windows Task Scheduler .

---

Another important place to find a huge amount of BizTalk related articles is the TechNet Wiki itself. The best entry point is BizTalk Server Resources on the TechNet Wiki.