MIME Messages in Lotusscript

After an Upgrade from Domino 9.0.1 FP4 to FP5 on Windows Server, our Domino server crashed frequently. It left no reason in the logs, just simply started from scratch and left an NSD File. We sent a PMR to IBM and finally found out, the crash could be reproduced with one certain agent.

After debugging a while, I found the reason, and, more important for me, I learned something about the Lotusscript NotesMIMEEntity Class.

What we want to do

Sometimes, as a company, you want to send mails from a user friendly looking account, like “We love our Customers<noreply@company.com>”Trust model 😉. Good idea. The issue is, that the Domino Server tends to send that mail with the sender name of the Notes-ID that signed the Agent code in a Field “Sent by:”. That’s also not exactly, what you want the recipients to see.

So, the existing idea was, for some time already, to choose a little workaround within Domino. Put the mail in “Mail.box” on your server, make up a MIME mail yourself by writing the MIME parts in your own code (not depending on the Server conversion) and then simply SAVE the document in mail.box, not SEND it. The router will take it from there, take your Sender address (which was made up with code) for given and send it to your customer. The method is described by Josef Hoetzl here, thank you.

CAVEAT: Before you take the following agent as your example, be aware, there’s code that crashes the server. The (for today) reliable version is at the end of this post.

So, here is a simple (but dangerous) agent:

Sub Initialize
On Error GoTo Errorhandler
Dim s As New NotesSession
Dim dbmailbox As NotesDatabase
Dim maildoc As NotesDocument
Dim body As NotesMIMEEntity
Dim header As NotesMIMEHeader
Dim child As NotesMIMEEntity
Dim stream As NotesStream
Dim sender As String
Dim send2Address As String
sender = "Yellowbleeders<notes@ibmlotus.com>"
send2Address = "Kermit_the_frog@theMuppets.com"
Set dbmailbox =  s.getDatabase(s.currentdatabase.server,"mail.box",False) 'opens the server's mailbox
'Make the new document
Set maildoc = New NotesDocument(dbmailbox)

 

Pretty easy until here, then we have to tell Domino, that we do all the MIME stuff ourselves.

s.Convertmime = False

That is a session property. We are supposed to reset it later. I’ve read other code for that property, like a switch/toggle version, but for simplicity, we stay here. Roland Praml has the ultimate Class here, thank you, Roland.

Now, on for the MIME part. Check Wikipedia for the different parts and specific order.

' Create the MIME Entity
Set body = maildoc.CreateMIMEEntity

'Create the MIME headers
Set header = body.CreateHeader({MIME-Version})
Call header.SetHeaderVal("1.0")

'Content Type
Set header = body.CreateHeader("Content-Type")
Call header.SetHeaderValAndParams({multipart/alternative;charset="UTF-8"})

'Subject
Set header = body.CreateHeader("Subject")
Call header.SetHeaderVal("Testmail")

'To
Set header = body.CreateHeader("To")
Call header.SetHeaderVal(send2Address)

'From (the lied adapted address) 
Set header = body.CreateHeader("From")
Call header.SetHeaderVal(sender)

'Recipients (see RFC 2822) 
'In RFC 2822 there is not "Recipients" header, but
'there is "other/optional" headers.
'We had it here because of a mistake
'This is the Killer for Fixpack 5! But the crash itself happens later 
'(Yellow dressed Uma Thurman) 

Set header = body.CreateHeader("Recipients")
Call header.SetHeaderVal(send2Address)


'  *** Headers set, begin Mail Body ***

'Send the plain text part first
Set child = body.createChildEntity()  'ChildEntities within the MIMEEntity
Set stream = s.createStream()
Call WritePlainText(stream) ' see later, content is made outside in a sub
Call child.setContentFromText(stream, {text/plain;charset="iso-8859-1"}, ENC_IDENTITY_8BIT)
Call stream.Truncate() ' ends the first child

'Send the HTML part.
Set child = body.createChildEntity()
Call WriteHTML(stream) ' same as with plain text, content is maintained outside
Call child.setContentFromText(stream, {text/html;charset="iso-8859-1"},ENC_IDENTITY_8BIT)
Call stream.Truncate()

 'Close the stream 
Call stream.Close()

'Close the whole MIME Part
Call maildoc.Closemimeentities(True' <--------------------- BOOOOOM! Tarantino.
' it turns out, that CloseMimeEntities does not like a header with the name 
' Recipients. You can add other names, but not Recipients. Recipients just lets 
' your server die. 

Now there is some Notes Fields. They are for the Domino Router task. It needs some information in fields, it seems.
I messed around with assigning the fields in the first part of the code, before I did the MIME stuff. But it turns out, that Domino, when you create some fields with keywords BEFORE you create the MIMEEntity, it automatically converts your field contents to MIME headers, hence ignoring your s.ConvertMIME = False. What happens next, if you create that header “again” in your code, your header pointer will be nothing and then, when you want to set it’s value, you get “Object variable not set”.

So if your code says:

Call maildoc.replaceItemvalue("Form", "Memo")
Call maildoc.replaceItemvalue("Subject", "NotesSubjectTest")

BEFORE you set

Set header = body.CreateHeader("Subject")
Call header.SetHeaderVal("Testmail")

then with Call header.SetHeaderVal(“Testmail”), it bails out.

The internal structure of the MIME Object looks like this, immediately after creating the mime entity from the example:

Bildschirmfoto 2016-02-17 um 16.03.38

so, after setting some headers, the following happens:

Bildschirmfoto 2016-02-17 um 16.09.06

 

1a shows a name for a list. The name is “HEADERS”. You can see in
1b, that list is a collection of Headerobject names, followed by a colon, and the content as string. If you look at
1c, you see the “Subject”-line with it’s content. Apparently, as seen in
1d, the list is separated with linefeeds, the height of that line is one line exactly, but you can use the arrows to go up and down.

Oh, and the Headerobjects (not open in that picture) contain the Name as String.

 

What happens, if you set your Notes fields before you begin your MIME? Some implicit fun.

Bildschirmfoto 2016-02-18 um 11.59.43

CreateMIMEEntity also looks at the underlying Notes Document and finds a few fields it takes as headers for MIME. It may be a philosophical question, if we want it to do that, given that we explicitly switched off MIME Conversion. But, nevertheless, this is what happens:

a) an existing Subject field is going to the MIME header “Subject”
b) an existing From field goes to a “From” header
c) BUT!!! an existing SendTo field is converted to a “To” header. BUT2: I tried it: if you make a “To” field, it DOES NOT convert into the “to” header. Don’t ask me, why. I could interpret a line in the help document about getting special support if you name your mime entity “body” (Just a wild guess: Home office here?) Did not try with more keywords, though.

(UPDATE3: Tested with a different name for the MIMEEntitiy, instead of “body” I took “myMIME”. The sequence of inheritance is exactly the same as described. There is no difference in handling depending on the name of the entity. My interpretation of documentation was wrong for this line :ENDOFUPDATE3)

The PROBLEM then is: If you create a header with body.CreateHeader(“Subject”) after that, the header will neither point to the existing header nor to a newly created header. It is NULL. When assigning a value with Call header.SetHeaderVal, you get “Object variable not set”.

The workaround: If you are aware of the issue, you could use Call header.AddValText(“This is my Umlaut Test äöüß”,”iso-8859-1″), where the second parameter is the iso character set. See Josef Hoetzl again for filename save character sets.
Safety would suggest, you used, if you are not sure you are alone, some code to test for existing headers. Just an example from help. There is a documentation at IBM: IBM Knowledge Center

Forall header In body.HeaderObjects
        ' some test code
      End Forall

 

UPDATE: The Fixpack 5 issue was recognized as a SPR and APAR and should be solved in the future.

Update 2: the easy working agent, as promised.

Update 3: Sven Petri pointed to an issue in his comment, I added the line needed below.

%REM
Agent RFC2822 safe version for Domino901FP5 and up-
Created Feb 15, 2016 by haha/saidtheclown
Description:

The agent creates a new document in mail.box of the current server.
it then creates a MimeEntity and sets some Headers, some content and stores the document,
but does not SEND it.
Thus, the Router picks up the doc and sends it with the sender we put in the FROM Header, not using
the signer of the agent, which would use if we SENT it.
%END REM
Option Public
Option Declare

Sub Initialize
On Error GoTo Errorhandler
Dim s As New NotesSession
Dim dbmailbox As NotesDatabase
Dim maildoc As NotesDocument
Dim body As NotesMIMEEntity
Dim header As NotesMIMEHeader
Dim child As NotesMIMEEntity
Dim stream As NotesStream
Dim sender As String
Dim send2Address As String

s.Convertmime = False

sender = "Yellowbleeders<notes@ibmlotus.com>"
send2Address = "kermit_the_frog@themuppets.com"
Set dbmailbox = s.getDatabase(s.currentdatabase.server,"mail.box",False)

'Make the new document
Set maildoc = New NotesDocument(dbmailbox)

' Create the MIME Entity

Set body = maildoc.CreateMIMEEntity
'Create the MIME headers
Set header = body.CreateHeader({MIME-Version})
Call header.SetHeaderVal("1.0")

'Content Type
Set header = body.CreateHeader("Content-Type")
Call header.SetHeaderValAndParams({multipart/alternative;charset="UTF-8"})
'Subject
Set header = body.CreateHeader("Subject")
Call header.SetHeaderVal("Testmail")
'To
Set header = body.CreateHeader("To")
Call header.SetHeaderVal(send2Address)
'From
Set header = body.CreateHeader("From")
Call header.SetHeaderVal(sender)
'SendTo
Set header = body.CreateHeader("SendTo") ' not really needed, but allowed
Call header.SetHeaderVal(send2Address)

' *** Headers set, begin Mail Body ***
'Send the plain text part first
Set child = body.createChildEntity() 'ChildEntities are containers
Set stream = s.createStream()
Call WritePlainText(stream)

Call child.setContentFromText(stream, {text/plain;charset="iso-8859-1"}, ENC_IDENTITY_8BIT)

'Send the HTML part.
Call stream.Truncate()

Set child = body.createChildEntity()

Call WriteHTML(stream)

Call child.setContentFromText(stream, {text/html;charset="iso-8859-1"},ENC_IDENTITY_8BIT)
'Close the stream and Send it
Call stream.Truncate()
Call stream.Close()

Call maildoc.Closemimeentities(True)

' Done with the MIME Stuff, now on to setting fields for Notes

Call maildoc.replaceItemvalue("Form", "Memo")
Call maildoc.replaceItemvalue("Subject", "NotesSubjectTest")
Call maildoc.replaceItemvalue("SendTo", send2Address)
Call maildoc.replaceItemvalue("From", sender)
Call maildoc.Replaceitemvalue("Recipients", send2Address) ' This is the Notes FIELD
'Update 3 20160711
Call maildoc.Replaceitemvalue("PostedDate",Now) ' see Sven Petri's reply
' End of Update 3
Call Maildoc.Save(True,False) 'just save, so spoofing works.

Exit Sub
ErrorHandler:
Print Erl, Error
End Sub

Sub WritePlainText(stream As NotesStream)

Dim Plain As String

Plain = "plain Content"

Call stream.WriteText(Plain , EOL_CRLF)

Done:
End Sub

Sub WriteHTML(stream As NotesStream)
Dim content As Variant

'BEGIN: Header
Call stream.WriteText(|<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">| & Chr(10) )
Call stream.WriteText(|<html>| & Chr(10))
Call stream.WriteText(|<head>| & Chr(10))

'END: HEADER
Call stream.WriteText (|</head>| & Chr(10) )

' BEGIN: HTML body
Call stream.WriteText (|<body>| & Chr(10) )
Call stream.WriteText (||) 
'BEGIN: Content Call stream.Writetext("HTML Content", 0) 
Call stream.WriteText (|| & Chr(10) ) 
Call stream.WriteText (|</body>| & Chr(10) ) 
' END: HTML bodydiv 
'Close the HTML 
Call stream.WriteText (|</html>| & Chr(10) ) 
End Sub

Advertisements

5 thoughts on “MIME Messages in Lotusscript

  1. Sven Petri

    If i use your 2nd code and send the Mail to the internet adress of a Notes-User, the mail opens as a draft in the Notes-Client. It is fine if it gets send to external mail-adresses like you did it in your example here.
    Server: 9.0.1 FP4 on Windows

    Do you have any ideas for this?

    Reply
    1. eemnee Post author

      That’s interesting, Sven, thank you. I think I have seen that once, but didn’t recognise the issue.
      I would have to test if adding a DeliveredDate item could help, when back at the PC.

      Reply
  2. Sven Petri

    Hello eemnee,
    you are right. My colleague and i found this IBM-Entry yesterday: http://www-01.ibm.com/support/docview.wss?uid=swg21240444 .
    The item in charge for this behaviour is “PostedDate”. I think it should be noted in every example code that shows how to save documents in the mail.box. I have seen quite a few and none of them did. You could be the 1st 🙂
    Thanks for your reply and your blog-post.

    Reply
  3. rpraml

    Hello eemnee,

    thanks for the credits… 😊
    I had a lot of fun (and crashes) with the mime items.
    It seems that the whole mime stuff is a kind of adapter that emulates the mime structure on a notes document.

    btw. s.convertMime controls just the behaviour, if mime content should be converted on first access to a “similar looking” rich text item.
    This is done for backward compatibility, because older applications can not handle mime items. They always expect to get a rich text item when calling doc.getFirstItem(“body”)

    We use mime entities in our applications as data Container for XML files and objects serialized in binary data.

    Unfortunately documents often get corrupt, if a third party product or custom agent does not set convertMime=false

    A simple access to “doc.items” triggers the conversion to richtext (which means that the content of the mime items are lost)
    Also the notes debugger triggers this conversion, if you expand a document in the variables view.

    Cheers Roland

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s