App: [HOLD for payment 2023-03-09] [$4000] Inconsistent formatting on multi-line formatted auto-email

The one thing we need to circle back on is the appropriate payment. Our auto-increasing logic pushed this issue up to $8,000, though at that time we had already agreed on the path forward, asking Mark to re-state his final proposal. As a result, I do think $4,000 is the more accurate compensation though I’m curious for others thoughts. cc @NikkiWines @thesahindia

Regression Test Proposal

Bug: Multi-line bold/italic/strikethrough emails render incorrectly

Proposed Test Steps:

1. Add the below as a new comment (paste as plain text):

~abc@gmail.com
def@gmail.com~
_abc@gmail.com
def@gmail.com_
*abc@gmail.com
def@gmail.com*

1. Verify that that each line is styled correctly:
   • Lines 1-2 are strikethrough
   • Lines 3-4 are italic
   • Lines 5-6 are bold
2. Verify that each line renders the full email as a mailto: link
3. Edit the comment and verify that it is the same as 1.

Do we agree 👍 or 👎

New job post here for good measure

Payment sent, contract ended @kerupuksambel!

Upwork job price has been updated to $4000

Regression Test Proposal

New tests were added in expensify-common to preserve the previous correct behavior not affected by this issue.

Proposed Test Steps:

1. Add the below as a new comment (paste as plain text):

~abc@gmail.com~
*abc@gmail.com*
_abc@gmail.com_
a~b@gmail.com
a*b@gmail.com
a_b@gmail.com

1. Verify that that each line is styled correctly:
   • Line 1 is strikethrough
   • Line 2 is bold
   • Line 3 is italic
   • Lines 4-6 are normal
2. Verify that each line renders the full email as a mailto: link
3. Edit the comment and verify that it is the same as 1.

Do we agree 👍 or 👎

Yeah let’s assign @marktoman and move forward.

Proposal

Please re-state the problem that we are trying to solve in this issue.

Auto-emails do not properly trim bold/italic/strikethrough MD characters on multi-line.

What is the root cause of that problem?

The regex was made for single-line emails.

What changes do you think we should make in order to solve the problem?

The regex needs to be updated so it reliably prevents the characters from being included for both single-line and multi-line. We achieve this by preventing the email from starting with the *_~ characters. The established consensus is that such emails should not be supported and the major email providers don’t support them either.

/(?![^<]*>|[^<>]*<\\)([_*~]*)([_*~]*?)([a-zA-Z0-9.!#$%&'+/=?^`{|}-][a-zA-Z0-9.!#$%&'*+/=?^_`{|}~-]*@[a-zA-Z0-9-]+?(\.[a-zA-Z]+)+)\2\1(?![^<]*(<\/pre>|<\/code>|<\/a>))/gim

What alternative solutions did you explore? (Optional)

None worth mentioning.

Going to hold on commenting so that @thesahindia can reply to @marktoman’s most recent comment first.

Also, IMO, we shouldn’t be going with RFC format for the email here. ~ in the email isn’t allowed by big players which means that eventually RFC will change as well. Allowing for ~ in the email adds a lot of complexity to our regex.

Ah no, do you mind getting that going? I’m working on some other R&D projects at the moment and I haven’t been able to circle back around to this.

Given that our app is still largely beta, we don’t need to worry too much about what users are used to. That said, this is a good one to bring out to discussion in our Slack community. I’ll circle back on that and CC those in this issue I can find using similar usernames to GH.

I should also note, one outstanding question is whether multi-line markdown should be supported. Clearly the issue right now is that we do support it, we just support it inconsistently. That said, I think it’s equally valid to question, should we stick to markdown within one line, before the break? I can only imagine this is one of several edge cases we’re going to face because we allow multi-line.