TypeError for SafeMIMEText #148

Stranger6667 · 2018-05-27T07:20:20Z

Follow up for this comment:

@Stranger6667 Is this working for you? I am getting the same exception with Python 2.7, Django 1.11 as well as Python 3.6, Django 1.11.

I get

TypeError: Object of type 'SafeMIMEText' is not JSON serializable
and I am unable to debug the issue.

Hi @suriya !
Could you, please, add some code to reproduce the issue?

suriya · 2018-05-27T09:31:39Z

Please take a look at this Github gist.

I am reproducing the Python code here. Run the code with Django version 1.11 and Python 3.6. I think the behavior should be the same for other versions as well.

The summary is that Postmarker crashes when sending an email with a HTML alternative as well as an attachment. Sending a mail with a HTML alternative alone works. Send a mail with an attachment alone works.

There is also a different issue. Sending a text/plain attachment also causes an error. It looks like Postmarker is not encoding the attachment body properly.

from django.conf import settings

settings.configure(
    DEBUG=True,
    SECRET_KEY='A-random-secret-key!',
    EMAIL_BACKEND = 'postmarker.django.EmailBackend',
    POSTMARK = {
        'TOKEN': 'POSTMARK_API_TEST',
        'TEST_MODE': True,
        'VERBOSITY': 3,
    }
)

def text_and_html_alternative_success():
    """
    Thus function sends a text body and html alternative.

    This function succeeds.
    """
    from django.core.mail import EmailMultiAlternatives
    msg = EmailMultiAlternatives(
        subject='Subject', body='Text body', from_email='[email protected]',
        to=['[email protected]']
    )
    msg.attach_alternative('<html></html>', "text/html")
    msg.send(fail_silently=False)

def text_and_pdf_attachment_success():
    """
    This functions sends a text body and PDF attachment

    This function succeeds.
    """
    from django.core.mail import EmailMultiAlternatives
    msg = EmailMultiAlternatives(
        subject='Subject', body='Body', from_email='[email protected]',
        to=['[email protected]']
    )
    msg.attach('hello.pdf', 'PDF-File-Contents', 'application/pdf')
    msg.send(fail_silently=False)

def text_html_alternative_and_pdf_attachment_failure():
    """
    This functions sends a text body, HTML alternative, and PDF attachment.

    This function fails.
    """
    from django.core.mail import EmailMultiAlternatives
    msg = EmailMultiAlternatives(
        subject='Subject', body='Body', from_email='[email protected]',
        to=['[email protected]']
    )
    msg.attach_alternative('<html></html>', "text/html")
    msg.attach('hello.pdf', 'PDF-File-Contents', 'application/pdf')
    msg.send(fail_silently=False)

def text_and_text_attachment_failure():
    """
    This function sends a text body and text attachment.

    This function fails.
    """
    from django.core.mail import EmailMultiAlternatives
    msg = EmailMultiAlternatives(
        subject='Subject', body='Body', from_email='[email protected]',
        to=['[email protected]']
    )
    msg.attach('hello.txt', 'Hello World', 'text/plain')
    msg.send(fail_silently=False)

if __name__ == '__main__':
    import django
    django.setup()
    # text_and_html_alternative_success()
    # text_and_pdf_attachment_success()
    # text_html_alternative_and_pdf_attachment_failure()
    # text_and_text_attachment_failure()

Stranger6667 · 2018-05-27T14:28:17Z

Regarding plain text attachment. You have to pass base64 encoded content:

From the docs

Note! Content should be encoded as Base64 string. Then pass the attachments to send()

Regarding the second issue - I'll take a look tomorrow

Thank you for reporting the problem :)

Stranger6667 · 2018-05-27T14:34:11Z

Probably, a validation for base64 string could be added. What do you think?

suriya · 2018-05-27T14:39:28Z

The issue is in deconstruct_multipart

postmarker/postmarker/models/emails.py

Line 80 in 62cc043

def deconstruct_multipart(message):

When passed a message with both an HTML alternative as well as an attachment, the function returns incorrect output. (That's what I think is going on).

suriya · 2018-05-27T14:45:50Z

I am a bit confused. Is the base64 encoding needed only for text/plain or for other types as well? I have sent PDF attachments by providing the raw content without any encoding.

Also, the base64 encoding could be a requirement for a low level API. However, it should not be required when sending emails from Django. (I don't need to send text attachments. I came across this issue while debugging the other TypeError issue). When sending from Django, the API to the user should be unchanged. The user should simply have to change their backend and the code should work. Any base-64 encoding should be performed by the postmarker library and not by the user.

suriya · 2018-05-27T18:10:57Z

The following patch seems to work for me. I don't know enough details to know if I am doing the right thing.

diff --git a/postmarker/models/emails.py b/postmarker/models/emails.py
index bb8a4c7..c629d95 100644
--- a/postmarker/models/emails.py
+++ b/postmarker/models/emails.py
@@ -76,21 +76,29 @@ def prepare_attachments(attachment):
         result = attachment
     return result
 
-
-def deconstruct_multipart(message):
-    text, html, attachments = None, None, []
-    for part in message.walk():
-        if part is message:
-            continue
-        content_type = part.get_content_type()
-        if content_type == 'text/plain' and text is None:
-            text = part.get_payload(decode=True).decode('utf8')
-        elif content_type == 'text/html' and html is None:
-            html = part.get_payload(decode=True).decode('utf8')
+def deconstruct_multipart_recursive(seen, text, html, attachments, message):
+    if (message in seen):
+        return
+    seen.add(message)
+    if isinstance(message, MIMEMultipart):
+        for part in message.walk():
+            deconstruct_multipart_recursive(seen, text, html, attachments, part)
+    else:
+        content_type = message.get_content_type()
+        if content_type == 'text/plain' and not text:
+            text.append(message.get_payload(decode=True).decode('utf8'))
+        elif content_type == 'text/html' and not html:
+            html.append(message.get_payload(decode=True).decode('utf8'))
         else:
-            attachments.append(part)
-    return text, html, attachments
+            attachments.append(message)
 
+def deconstruct_multipart(message):
+    seen = set()
+    text = []
+    html = []
+    attachments = []
+    deconstruct_multipart_recursive(seen, text, html, attachments, message)
+    return ((text and text[0]) or None, (html and html[0]) or None, attachments)
 
 class BaseEmail(Model):
 
@@ -178,10 +186,7 @@ class Email(BaseEmail):
         :param manager: :py:class:`EmailManager` instance.
         :return: :py:class:`Email`
         """
-        if isinstance(message, MIMEMultipart):
-            text, html, attachments = deconstruct_multipart(message)
-        else:
-            text, html, attachments = message.get_payload(decode=True).decode('utf8'), None, []
+        text, html, attachments = deconstruct_multipart(message)
         subject = prepare_header(message['Subject'])
         sender = prepare_header(message['From'])
         to = prepare_header(message['To'])

Fixes issue Stranger6667#148

* Handle emails with HTML alternatives and attachments Fixes issue #148 * Fix lint

Stranger6667 · 2018-06-05T20:53:07Z

Btw, regarding base64 - Django automatically encodes content in base64, that's why it was possible to send pdf without an error.

I'll make a change in postmarker to automatically encode data in base64 in similar way, but, probably it will be controlled by some kwarg, will see.

Stranger6667 · 2018-06-05T21:29:08Z

I am a bit confused. Is the base64 encoding needed only for text/plain or for other types as well?

I don't know yet :(

Stranger6667 · 2018-06-05T21:30:16Z

I'll create a separate issue for base64 encoding

suriya added a commit to suriya/postmarker that referenced this issue May 28, 2018

Handle emails with HTML alternatives and attachments

410baf3

Fixes issue Stranger6667#148

suriya mentioned this issue May 28, 2018

Handle emails with HTML alternatives and attachments #149

Merged

Stranger6667 pushed a commit to suriya/postmarker that referenced this issue Jun 5, 2018

Handle emails with HTML alternatives and attachments

a5e8790

Fixes issue Stranger6667#148

Stranger6667 pushed a commit that referenced this issue Jun 5, 2018

Handle emails with HTML alternatives and attachments (#149)

90ad813

* Handle emails with HTML alternatives and attachments Fixes issue #148 * Fix lint

Stranger6667 closed this as completed Jun 5, 2018

Stranger6667 mentioned this issue Jun 5, 2018

Automatically encode data to base64 if needed #153

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TypeError for SafeMIMEText #148

TypeError for SafeMIMEText #148

Stranger6667 commented May 27, 2018

suriya commented May 27, 2018

Stranger6667 commented May 27, 2018

Stranger6667 commented May 27, 2018

suriya commented May 27, 2018

suriya commented May 27, 2018

suriya commented May 27, 2018

Stranger6667 commented Jun 5, 2018

Stranger6667 commented Jun 5, 2018

Stranger6667 commented Jun 5, 2018

TypeError for SafeMIMEText #148

TypeError for SafeMIMEText #148

Comments

Stranger6667 commented May 27, 2018

suriya commented May 27, 2018

Stranger6667 commented May 27, 2018

Stranger6667 commented May 27, 2018

suriya commented May 27, 2018

suriya commented May 27, 2018

suriya commented May 27, 2018

Stranger6667 commented Jun 5, 2018

Stranger6667 commented Jun 5, 2018

Stranger6667 commented Jun 5, 2018