MIME Messages in Lotusscript

After an Upgrade from Domino 9.0.1 FP4 to FP5 on Windows Server, our Domino server crashed frequently. It left no reason in the logs, just simply started from scratch and left an NSD File. We sent a PMR to IBM and finally found out, the crash could be reproduced with one certain agent.

After debugging a while, I found the reason, and, more important for me, I learned something about the Lotusscript NotesMIMEEntity Class.

What we want to do

Sometimes, as a company, you want to send mails from a user friendly looking account, like “We love our Customers<noreply@company.com>”Trust model 😉. Good idea. The issue is, that the Domino Server tends to send that mail with the sender name of the Notes-ID that signed the Agent code in a Field “Sent by:”. That’s also not exactly, what you want the recipients to see.

So, the existing idea was, for some time already, to choose a little workaround within Domino. Put the mail in “Mail.box” on your server, make up a MIME mail yourself by writing the MIME parts in your own code (not depending on the Server conversion) and then simply SAVE the document in mail.box, not SEND it. The router will take it from there, take your Sender address (which was made up with code) for given and send it to your customer. The method is described by Josef Hoetzl here, thank you.

CAVEAT: Before you take the following agent as your example, be aware, there’s code that crashes the server. The (for today) reliable version is at the end of this post.

So, here is a simple (but dangerous) agent:

Sub Initialize
On Error GoTo Errorhandler
Dim s As New NotesSession
Dim dbmailbox As NotesDatabase
Dim maildoc As NotesDocument
Dim body As NotesMIMEEntity
Dim header As NotesMIMEHeader
Dim child As NotesMIMEEntity
Dim stream As NotesStream
Dim sender As String
Dim send2Address As String
sender = "Yellowbleeders<notes@ibmlotus.com>"
send2Address = "Kermit_the_frog@theMuppets.com"
Set dbmailbox =  s.getDatabase(s.currentdatabase.server,"mail.box",False) 'opens the server's mailbox
'Make the new document
Set maildoc = New NotesDocument(dbmailbox)

 

Pretty easy until here, then we have to tell Domino, that we do all the MIME stuff ourselves.

s.Convertmime = False

That is a session property. We are supposed to reset it later. I’ve read other code for that property, like a switch/toggle version, but for simplicity, we stay here. Roland Praml has the ultimate Class here, thank you, Roland.

Now, on for the MIME part. Check Wikipedia for the different parts and specific order.

' Create the MIME Entity
Set body = maildoc.CreateMIMEEntity

'Create the MIME headers
Set header = body.CreateHeader({MIME-Version})
Call header.SetHeaderVal("1.0")

'Content Type
Set header = body.CreateHeader("Content-Type")
Call header.SetHeaderValAndParams({multipart/alternative;charset="UTF-8"})

'Subject
Set header = body.CreateHeader("Subject")
Call header.SetHeaderVal("Testmail")

'To
Set header = body.CreateHeader("To")
Call header.SetHeaderVal(send2Address)

'From (the lied adapted address) 
Set header = body.CreateHeader("From")
Call header.SetHeaderVal(sender)

'Recipients (see RFC 2822) 
'In RFC 2822 there is not "Recipients" header, but
'there is "other/optional" headers.
'We had it here because of a mistake
'This is the Killer for Fixpack 5! But the crash itself happens later 
'(Yellow dressed Uma Thurman) 

Set header = body.CreateHeader("Recipients")
Call header.SetHeaderVal(send2Address)


'  *** Headers set, begin Mail Body ***

'Send the plain text part first
Set child = body.createChildEntity()  'ChildEntities within the MIMEEntity
Set stream = s.createStream()
Call WritePlainText(stream) ' see later, content is made outside in a sub
Call child.setContentFromText(stream, {text/plain;charset="iso-8859-1"}, ENC_IDENTITY_8BIT)
Call stream.Truncate() ' ends the first child

'Send the HTML part.
Set child = body.createChildEntity()
Call WriteHTML(stream) ' same as with plain text, content is maintained outside
Call child.setContentFromText(stream, {text/html;charset="iso-8859-1"},ENC_IDENTITY_8BIT)
Call stream.Truncate()

 'Close the stream 
Call stream.Close()

'Close the whole MIME Part
Call maildoc.Closemimeentities(True' <--------------------- BOOOOOM! Tarantino.
' it turns out, that CloseMimeEntities does not like a header with the name 
' Recipients. You can add other names, but not Recipients. Recipients just lets 
' your server die. 

Now there is some Notes Fields. They are for the Domino Router task. It needs some information in fields, it seems.
I messed around with assigning the fields in the first part of the code, before I did the MIME stuff. But it turns out, that Domino, when you create some fields with keywords BEFORE you create the MIMEEntity, it automatically converts your field contents to MIME headers, hence ignoring your s.ConvertMIME = False. What happens next, if you create that header “again” in your code, your header pointer will be nothing and then, when you want to set it’s value, you get “Object variable not set”.

So if your code says:

Call maildoc.replaceItemvalue("Form", "Memo")
Call maildoc.replaceItemvalue("Subject", "NotesSubjectTest")

BEFORE you set

Set header = body.CreateHeader("Subject")
Call header.SetHeaderVal("Testmail")

then with Call header.SetHeaderVal(“Testmail”), it bails out.

The internal structure of the MIME Object looks like this, immediately after creating the mime entity from the example:

Bildschirmfoto 2016-02-17 um 16.03.38

so, after setting some headers, the following happens:

Bildschirmfoto 2016-02-17 um 16.09.06

 

1a shows a name for a list. The name is “HEADERS”. You can see in
1b, that list is a collection of Headerobject names, followed by a colon, and the content as string. If you look at
1c, you see the “Subject”-line with it’s content. Apparently, as seen in
1d, the list is separated with linefeeds, the height of that line is one line exactly, but you can use the arrows to go up and down.

Oh, and the Headerobjects (not open in that picture) contain the Name as String.

 

What happens, if you set your Notes fields before you begin your MIME? Some implicit fun.

Bildschirmfoto 2016-02-18 um 11.59.43

CreateMIMEEntity also looks at the underlying Notes Document and finds a few fields it takes as headers for MIME. It may be a philosophical question, if we want it to do that, given that we explicitly switched off MIME Conversion. But, nevertheless, this is what happens:

a) an existing Subject field is going to the MIME header “Subject”
b) an existing From field goes to a “From” header
c) BUT!!! an existing SendTo field is converted to a “To” header. BUT2: I tried it: if you make a “To” field, it DOES NOT convert into the “to” header. Don’t ask me, why. I could interpret a line in the help document about getting special support if you name your mime entity “body” (Just a wild guess: Home office here?) Did not try with more keywords, though.

(UPDATE3: Tested with a different name for the MIMEEntitiy, instead of “body” I took “myMIME”. The sequence of inheritance is exactly the same as described. There is no difference in handling depending on the name of the entity. My interpretation of documentation was wrong for this line :ENDOFUPDATE3)

The PROBLEM then is: If you create a header with body.CreateHeader(“Subject”) after that, the header will neither point to the existing header nor to a newly created header. It is NULL. When assigning a value with Call header.SetHeaderVal, you get “Object variable not set”.

The workaround: If you are aware of the issue, you could use Call header.AddValText(“This is my Umlaut Test äöüß”,”iso-8859-1″), where the second parameter is the iso character set. See Josef Hoetzl again for filename save character sets.
Safety would suggest, you used, if you are not sure you are alone, some code to test for existing headers. Just an example from help. There is a documentation at IBM: IBM Knowledge Center

Forall header In body.HeaderObjects
        ' some test code
      End Forall

 

UPDATE: The Fixpack 5 issue was recognized as a SPR and APAR and should be solved in the future.

Update 2: the easy working agent, as promised.

Update 3: Sven Petri pointed to an issue in his comment, I added the line needed below.

%REM
Agent RFC2822 safe version for Domino901FP5 and up-
Created Feb 15, 2016 by haha/saidtheclown
Description:

The agent creates a new document in mail.box of the current server.
it then creates a MimeEntity and sets some Headers, some content and stores the document,
but does not SEND it.
Thus, the Router picks up the doc and sends it with the sender we put in the FROM Header, not using
the signer of the agent, which would use if we SENT it.
%END REM
Option Public
Option Declare

Sub Initialize
On Error GoTo Errorhandler
Dim s As New NotesSession
Dim dbmailbox As NotesDatabase
Dim maildoc As NotesDocument
Dim body As NotesMIMEEntity
Dim header As NotesMIMEHeader
Dim child As NotesMIMEEntity
Dim stream As NotesStream
Dim sender As String
Dim send2Address As String

s.Convertmime = False

sender = "Yellowbleeders<notes@ibmlotus.com>"
send2Address = "kermit_the_frog@themuppets.com"
Set dbmailbox = s.getDatabase(s.currentdatabase.server,"mail.box",False)

'Make the new document
Set maildoc = New NotesDocument(dbmailbox)

' Create the MIME Entity

Set body = maildoc.CreateMIMEEntity
'Create the MIME headers
Set header = body.CreateHeader({MIME-Version})
Call header.SetHeaderVal("1.0")

'Content Type
Set header = body.CreateHeader("Content-Type")
Call header.SetHeaderValAndParams({multipart/alternative;charset="UTF-8"})
'Subject
Set header = body.CreateHeader("Subject")
Call header.SetHeaderVal("Testmail")
'To
Set header = body.CreateHeader("To")
Call header.SetHeaderVal(send2Address)
'From
Set header = body.CreateHeader("From")
Call header.SetHeaderVal(sender)
'SendTo
Set header = body.CreateHeader("SendTo") ' not really needed, but allowed
Call header.SetHeaderVal(send2Address)

' *** Headers set, begin Mail Body ***
'Send the plain text part first
Set child = body.createChildEntity() 'ChildEntities are containers
Set stream = s.createStream()
Call WritePlainText(stream)

Call child.setContentFromText(stream, {text/plain;charset="iso-8859-1"}, ENC_IDENTITY_8BIT)

'Send the HTML part.
Call stream.Truncate()

Set child = body.createChildEntity()

Call WriteHTML(stream)

Call child.setContentFromText(stream, {text/html;charset="iso-8859-1"},ENC_IDENTITY_8BIT)
'Close the stream and Send it
Call stream.Truncate()
Call stream.Close()

Call maildoc.Closemimeentities(True)

' Done with the MIME Stuff, now on to setting fields for Notes

Call maildoc.replaceItemvalue("Form", "Memo")
Call maildoc.replaceItemvalue("Subject", "NotesSubjectTest")
Call maildoc.replaceItemvalue("SendTo", send2Address)
Call maildoc.replaceItemvalue("From", sender)
Call maildoc.Replaceitemvalue("Recipients", send2Address) ' This is the Notes FIELD
'Update 3 20160711
Call maildoc.Replaceitemvalue("PostedDate",Now) ' see Sven Petri's reply
' End of Update 3
Call Maildoc.Save(True,False) 'just save, so spoofing works.

Exit Sub
ErrorHandler:
Print Erl, Error
End Sub

Sub WritePlainText(stream As NotesStream)

Dim Plain As String

Plain = "plain Content"

Call stream.WriteText(Plain , EOL_CRLF)

Done:
End Sub

Sub WriteHTML(stream As NotesStream)
Dim content As Variant

'BEGIN: Header
Call stream.WriteText(|<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">| & Chr(10) )
Call stream.WriteText(|<html>| & Chr(10))
Call stream.WriteText(|<head>| & Chr(10))

'END: HEADER
Call stream.WriteText (|</head>| & Chr(10) )

' BEGIN: HTML body
Call stream.WriteText (|<body>| & Chr(10) )
Call stream.WriteText (||) 
'BEGIN: Content Call stream.Writetext("HTML Content", 0) 
Call stream.WriteText (|| & Chr(10) ) 
Call stream.WriteText (|</body>| & Chr(10) ) 
' END: HTML bodydiv 
'Close the HTML 
Call stream.WriteText (|</html>| & Chr(10) ) 
End Sub

			

ConnectoSphere 2015 App “TUTU”

Mat Newman approached Vitor Pereira and me two years ago to maintain the ConnectoSphere Totally Unofficial Totally Unsupported Session planning tool on Notes Basis. (TUTU)

That worked fine last year (2014) and even better this year.

I added a few features, and here’s how it works:

a) get the Database from Mat here.

—– EDIT ——–
if you cannot reach Mat Newmans server (it seems to reset connections in some places), you can get the file here: it’s on google drive, thus the mega url:

https://drive.google.com/file/d/0BwMuQonT61SWNGZsWVJtNnJ2ak0/view?usp=sharing
—– EDIT END ——

b) once you have it local, synchronize with the server

c) If you only use Notes Mail/Calendar, everything is fine.

d) if you want to have your schedule on another calendar (iCal type) too, edit your Preferences, there is a link in the Outline on the left.

outline

the Outline for your iCal Preferences

here we go:

Profile

e) Select your Sessions you want to schedule.

Session list

and “Add selected to my Calendar”. Wait a bit. After  some seconds you have calendar entries in your Notes Calendar (in the right timezone, so don’t panic, if the entries are round midnight).
And if you happen to have a managed Mailfile, it will take a few minutes to update the display from your server, so, again, relax.

If you send an email with the iCal data, you can see a statement in the Statusline:

Status message it means “Mail sent”

g) you may then also find a mail in your extra mail client:

mail

you can import that file in your other calendar, and bingo, happy. yeah.

Meet you at Lotusphere (Lastusphere), erm, IBMConnectED 2015

Die maximale Kapazität zum Speichern von Mailregeln wurde möglicherweise überschritten.

Mal einer auf deutsch, weil ich die Meldung nicht auf deutsch gefunden habe. Und sie taucht auch außerhalb des früher beschriebenen SWIFT-Problems auftaucht.

Festgestellt in: Notes 9+

Symptom:

Bildschirmfoto 2014-11-12 um 07.48.04

Diese Meldung führt einen gewaltig in die Irre.

Der Hintergrund:

Man darf ja in Notes auf andere Mailfiles und Kalender zugreifen, wenn die Besitzer das erlauben. Meist allerdings nur mit LESENDEM Zugriff. Das klappt auch ganz gut.

Nun gibt es aber Funktionen – eine davon ist der “Tag auf einen Blick” aus der Seitenleiste -, die würden sich gern, damit sie sich später schneller erinnern können, ein paar interne Notizen in diese Mailfiles oder Kalender schreiben. Das dürfen sie allerdings nicht, und deshalb kommt es zu einem “Du darfst nicht schreiben”-Fehler. Der wird in diesem Programmteil dann einfach vom Programm selbst falsch interpretiert und als “Du darfst keine Mailregeln mehr schreiben, deshalb glaube ich, der Speicher ist erschöpft, und damit du dich ärgerst, lass ich dich auch noch eine Litanei lesen, bis du dich gar nicht mehr auskennst” ausgegeben.

Tja, auch Programmierer machen tatsächlich Fehler.

Was kann man tun? Einfach mit OK bestätigen, dann schauen, welche zusätzlichen Kalender/Mailfiles man offen hat, diese schließen. -> Fehler futsch.

Domino Server Console on Windows Server 2012 R2 (Updated)

After some looong time working with Domino, IT decided to go up to R901 and have XPages on our servers.
No, not Linux, we go with Windows Server 2012 R2. Well.

Network team installed the Domino Software, started it. But Domino would not start up as a service, though configured. oops. And Domino would not even show the server console. And there was no console.log or other log file in the IBM_TECHNICAL_SUPPORT directory.

The ServerConsole could not connect to the server. “There ain’t no server on port 2050”

Interesting things to watch were:
A Ping to the server name brought back ::1: which is the loopback address of the machine in IP V6 (think of 127.0.0.1). Even though IPV6 was NOT configured on the Network interfaces. (Windows? Knock Knock. Anybody in there?)

A Ping machine_name -4 brought back a V4 IP. (For better understanding, let it be 192.168.0.200 for this example)

We added

ServerController=1
TCPIP_ControllerTcpIpAddress=192.168.0.200:2050

to our notes.ini and restarted the DOMINO Server.
That’s the lines the ServerConsole Client uses to connect to the ServerConsole Controller (that’s the counterpart running in Domino on standard port 2050).

Hey, it worked!

But.
After rebooting the Server machine (the whole machine, not only Domino), it again did not work.

We found a recommendation to delete the “dconsole.ini” in the Notes Data directory (it holds configuration for the ServerConsole Controller) did not help.

We then edited the line
LocalHost=COMMONSERVERNAME(FQDN):2050
and changed it to
LocalHost=192.168.0.200:2050

But, whenever we started Domino, it switched the line back to:
LocalHost=COMMONSERVERNAME(FQDN):2050 and that Common Server name was an official Server.Domain.Countrycode name, so brought back an address from the DNS, and the address itself was of course in an outside range. The machine did not have access to that external range. And the Network Team did not want to give access to an outside range.

A new ping server name -4 also revealed: the server had a new internal IP address!

The cause: For whatever reason, our lovely server has 4 network interfaces configured. Network team needs that. But Windows Server 2012 will not prioritize these addresses on startup, or better: does prioritize the addresses in a special way (that looks random at each server start), i.e. we cannot use that way.

There is a workaround we made: John Biswell: set primary IP Address Thank you John Biswell.

Whenever the Domino Server started, it switched the IP Address in the LocalHost-Line to the Common Server Name. This name is obviously taken from the Fully Qualified Internet Hostname Field in the Basic section of the server configuration document. We could not change that for another reason, so we needed a workaround.

Here is the manual configuration that works for us:

We set the same primary IP in the above mentioned TCPIP_ControllerTcpAddress line in Notes.ini:

TCPIP_ControllerTcpIpAddress=192.168.0.200:2050

Then added a line to the hosts file with the same address and the common server name.
192.168.0.200 server.domain.countrycode

Now the server can find itself, even on the console.
Of course, the workaround will hinder any browser or tool to access the outside hostname, but we can live with that on a Domino Server.

“Shame on the Domino Server”, they said. Ha!

Background info from IBM (the only hint you can find in a log):
Either the Server Controller is not running on the host INMAIL01/Acme or is not listening on port 2050

Hope it helps

Other mail constraints

Notes (IBM Notes now) allows a user to add other mail files to look into, or other calendars.
Security is managed by user themselves, granting you access to their mail file. One would think, the feature is well thought through, but on at least two occasions, you might get into trouble and at first hand might not see the reason.

case SWIFT File:
if you have installed Swift File (you know, the “guessing tool” that guesses the folder you might want to put your new mail into in order to forget it ;-)), and enabled your mail file to use Swift. In this case, no problem.

case SWIFT File and Access to other mail/calendar:
better enable all mail files you have access to for Swift. If you don’t, probability is high that your client crashes when you try to close it (at the end of a day, you want to leave and not wait for NSD to write the whole complaint to disk). Or don’t add other mail or calendar. People are not often aware of the fact that they have links to other mail on their workspace just because of a quick look in somebodies calendar.

case Quota:
If you have access to another mail file and your mail policy says “Show Quota Indicators on client” (you can find that in the Desktop Policy, tab “Mail”) then Notes seems to try to update something in each mail file you open, but you only have reader access. It then says something like “too many mail rules or you are you are over quota”. More verbose, of course, and misleading.

The german version:

MailError

Mail Error on wrong quota info

For people using other mail either give them a higher access level to the targeted mail file or kick the policy. Access level is usually done in “More.Preferences.AccessAndDelegation” and mostly sets readers only. You can give other people the right to edit, which would enable them to use the mail file.

Edit: IBM support found the code really quick after a small hint.