Updates in install scripts

README.md

@@ -6,11 +6,10 @@
 
 ## Features
 
-* Subscriber list management
+* Subscriber lists management
 * List segmentation
 * Custom fields
 * Email templates (including MJML-based templates)
-* Large CSV list import files
 * Custom reports
 * Automation (triggered and RSS campaigns)
 * Multiple users with granular user permissions and flexible sharing
@@ -20,6 +19,113 @@
 * 1 vCPU
 * 2048 MB RAM
 
+## Quick Start
+
+### Preparation
+Mailtrain creates three URL endpoints, which are referred to as "trusted", "sandbox" and "public". This allows Mailtrain
+to guarantee security and avoid XSS attacks in the multi-user settings. The function of these three endpoints is as follows:
+- *trusted* - This is the main endpoint for the UI that a logged-in user uses to manage lists, send campaigns, etc.
+- *sandbox* - This is an endpoint not directly visible to a user. It is used to host WYSIWYG template editors.
+- *public* - This is an endpoint for subscribers. It is used to host subscription management forms, files and archive.
+
+The recommended deployment of Mailtrain would use 3 DNS entries that all points to the **same** IP address. For example as follows:
+- *lists.example.com* - public endpoint (A record `lists` under `example.com` domain)
+- *mailtrain.example.com* - trusted endpoint (CNAME record `mailtrain` under `example.com` domain that points to `lists`)
+- *sbox.mailtrain.example.com* - sandbox endpoint (CNAME record `sbox.mailtrain` under `example.com` domain that points to `lists`)
+
+
+
+### Installation on fresh CentOS 7 (public website secured by SSL)
+
+This will setup a publicly accessible Mailtrain instance. All endpoints (trusted, sandbox, public) will provide both HTTP (on port 80)
+and HTTPS (on port 443). The HTTP ports just issue HTTP redirect to their HTTPS counterparts. The script below will also acquire a
+valid certificate from [Let's Encrypt](https://letsencrypt.org/).
+
+1. Login as root
+    ```
+    sudo -i
+    ```
+
+2. Install git
+    ```
+    yum install -y git
+    ```
+
+3. Download Mailtrain using git to the `/opt/mailtrain` directory
+    ```
+    cd /opt
+    git clone https://github.com/Mailtrain-org/mailtrain.git
+    cd mailtrain
+    git checkout development
+    ```
+
+4. Run the installation script. Replace the urls and your email address with the correct values. **NOTE** that running this script you agree
+   Let's Encrypt's conditions.
+    ```
+    sh setup/install-centos7-https.sh mailtrain.example.com sbox.mailtrain.example.com lists.example.com admin@example.com
+    ```
+
+5. Start Mailtrain and enable to be started by default when your server starts.
+    ```
+    systemctl start mailtrain
+    systemctl enable mailtrain
+    ```
+
+6. Open the trusted endpoint (like `https://mailtrain.example.com`)
+
+7. Authenticate as `admin`:`test`
+
+8. Update your password under admin/Account
+
+9. Update your settings under Administration/Global Settings.
+
+10. If you intend to sign your email by DKIM, set the DKIM key and DKIM selector under Administration/Send Configurations.
+
+
+### Installation on fresh CentOS 7 (local installation)
+
+This will setup a locally accessible Mailtrain instance (primarily for development and testing).
+All endpoints (trusted, sandbox, public) will provide only HTTP as follows:
+- http://localhost:3000 - trusted endpoint
+- http://localhost:3003 - sandbox endpoint
+- http://localhost:3004 - public endpoint
+
+1. Login as root
+    ```
+    sudo -i
+    ```
+
+2. Install git
+    ```
+    yum install -y git
+    ```
+
+3. Download Mailtrain using git to the `/opt/mailtrain` directory
+    ```
+    cd /opt
+    git clone https://github.com/Mailtrain-org/mailtrain.git
+    cd mailtrain
+    git checkout development
+    ```
+
+4. Run the installation script. Replace the urls and your email address with the correct values. **NOTE** that running this script you agree
+   Let's Encrypt's conditions.
+    ```
+    sh setup/install-centos7-local.sh
+    ```
+
+5. Start Mailtrain and enable to be started by default when your server starts.
+    ```
+    systemctl start mailtrain
+    systemctl enable mailtrain
+    ```
+
+6. Open the trusted endpoint http://localhost:3000
+
+7. Authenticate as `admin`:`test`
+
+
+
 ## Quick Start - Deploy with Docker
 #### Requirements:
 
@@ -38,23 +144,6 @@ Depending on how you have configured your system and Docker you may need to prep
 * Navigate to [http://localhost:3000/settings](http://localhost:3000/settings) and update service configuration.
 * Navigate to [http://localhost:3000/users/account](http://localhost:3000/users/account) and update user information and password.
 
-## Quick Start - Manual Install (any OS that supports Node.js)
-
-### Requirements: 
- * Mailtrain requires at least **Node.js v10**.
-
-  1. Download Mailtrain files using git: `git clone git://github.com/Mailtrain-org/mailtrain.git` (or download [zipped repo](https://github.com/Mailtrain-org/mailtrain/archive/master.zip)) and open Mailtrain folder `cd mailtrain`
-  2. Run `npm install --production` in the Mailtrain folder to install required dependencies
-  3. Copy [config/default.toml](config/default.toml) as `config/production.toml` and update MySQL and any other settings in it
-  4. Run the server `NODE_ENV=production npm start`
-  5. Open [http://localhost:3000/](http://localhost:3000/)
-  6. Authenticate as `admin`:`test`
-  7. Navigate to [http://localhost:3000/settings](http://localhost:3000/settings) and update service configuration
-  8. Navigate to [http://localhost:3000/users/account](http://localhost:3000/users/account) and update user information and password
-
-## Read The Docs
-For more information, please [read the docs](http://docs.mailtrain.org/).
-
 
 ## License
 

locales/en-US/common.json

@@ -203,50 +203,16 @@
   "triggered": "Triggered",
   "campaigns": "Campaigns",
   "campaignStatistics": "Campaign Statistics",
-  "total": "Total",
+  "subscriptionHasToBeSelectedToShowThe": "Subscription has to be selected to show the campaign for a test user.",
-  "delivered": "Delivered",
-  "blacklisted": "Blacklisted",
-  "bounced": "Bounced",
-  "complaints": "Complaints",
-  "unsubscribed": "Unsubscribed",
-  "opened": "Opened",
-  "clicked": "Clicked",
-  "uniqueVisitors": "Unique visitors",
-  "totalClicks": "Total clicks",
-  "campaignLinks": "Campaign links",
   "subscriptionId": "Subscription ID",
   "listId": "List ID",
   "listNamespace": "List namespace",
-  "opensCount": "Opens count",
-  "countries": "Countries",
-  "devices": "Devices",
-  "desktop": "Desktop",
-  "tv": "TV",
-  "tablet": "Tablet",
-  "phone": "Phone",
-  "bot": "Bot",
-  "car": "Car",
-  "console": "Console",
-  "distributionByDeviceType": "Distribution by device type",
-  "loadingChart": "Loading chart",
-  "deviceType": "Device type",
-  "count": "Count",
-  "unknown": "Unknown",
-  "distributionByCountry": "Distribution by country",
-  "country": "Country",
-  "detailedStatistics": "Detailed Statistics",
-  "listOfSubscribersThatOpenedTheCampaign": "List of subscribers that opened the campaign",
-  "subscriptionHasToBeSelectedToShowThe": "Subscription has to be selected to show the campaign for a test user.",
   "previewCampaignAs": "Preview campaign as",
   "preview": "Preview",
   "dateMustNotBeEmpty": "Date must not be empty",
   "dateIsInvalid": "Date is invalid",
   "timeMustNotBeEmpty": "Time must not be empty",
   "timeIsInvalid": "Time is invalid",
-  "confirmReset": "Confirm reset",
-  "doYouWantToResetTheCampaign?All": "Do you want to reset the campaign? All statistics and the track of delivered messages will be lost.",
-  "no": "No",
-  "yes": "Yes",
   "subscribers-1": "subscribers",
   "sendStatus": "Send status",
   "campaignIsScheduledForDelivery": "Campaign is scheduled for delivery.",
@@ -270,6 +236,7 @@
   "yourCampaignIsEnabledAndSendingMessages": "Your campaign is enabled and sending messages.",
   "disable": "Disable",
   "campaignStatus": "Campaign Status",
+  "computing": "computing ...",
   "targetListssegments": "Target lists/segments",
   "ifANewEntryIsFoundFromCampaignFeedANew": "If a new entry is found from campaign feed a new subcampaign is created of that entry and it will be listed here",
   "sendingTestEmail": "Sending test email",
@@ -284,12 +251,6 @@
   "inactive": "Inactive",
   "active": "Active",
   "campaignName": "Campaign \"{{name}}\"",
-  "deliveredEmails": "Delivered Emails",
-  "complained": "Complained",
-  "subscribersThatComplained": "Subscribers that Complained",
-  "emailsThatBounced": "Emails that Bounced",
-  "subscribersThatUnsubscribed": "Subscribers that Unsubscribed",
-  "clicks": "Clicks",
   "theseFilesArePubliclyAvailableViaHttpSo": "These files are publicly available via HTTP so that they can be linked to from the content of the campaign.",
   "theseFilesWillBeAttachedToTheCampaign": "These files will be attached to the campaign emails as proper attachments. This means they count towards to the eventual size of the email.",
   "triggerName": "Trigger \"{{name}}\"",
@@ -309,8 +270,13 @@
   "selectTheEventThatTriggersSendingThe": "Select the event that triggers sending the campaign.",
   "enabled": "Enabled",
   "daysAfter": "Days after",
+  "yes": "Yes",
+  "no": "No",
   "latestOpen": "Latest open",
   "latestClick": "Latest click",
+  "delivered": "Delivered",
+  "opened": "Opened",
+  "clicked": "Clicked",
   "notOpened": "Not opened",
   "notClicked": "Not clicked",
   "close": "Close",
@@ -373,7 +339,7 @@
   "createList": "Create List",
   "thisIsTheListIdDisplayedToTheSubscribers": "This is the list ID displayed to the subscribers",
   "contactEmail": "Contact email",
-  "contactEmailShownInTheListSubscription": "Contact email shown in the list subscription and management forms. If no contact email is given, the admin email from Global settings is used.",
+  "contactEmailUsedInSubscriptionFormsAnd": "Contact email used in subscription forms and emails that are sent out. If not filled in, the admin email from the global settings will be used.",
   "homepage": "Homepage",
   "homepageUrlUsedInSubscriptionFormsAnd": "Homepage URL used in subscription forms and emails that are sent out. If not filled in, the default homepage from global settings will be used.",
   "recipientsNameTemplate": "Recipients name template",
@@ -485,8 +451,6 @@
   "formsPreview": "Forms Preview",
   "listToPreviewOn": "List To Preview On",
   "selectListWhoseFieldsWillBeUsedToPreview": "Select list whose fields will be used to preview the forms.",
-  "noteTheseLinksAreSolelyForAQuickPreview": "Note: These links are solely for a quick preview. To get the address of the subscription form, go to the list\\'s subscribers and click on \"Subscription Form\".",
-  "formPreview": "Form preview:",
   "templates": "Templates",
   "customFormsUseMjmlForFormattingSeeThe": "Custom forms use MJML for formatting. See the MJML documentation <1>here</1>",
   "createCustomForm": "Create Custom Form",
@@ -664,6 +628,7 @@
   "testUser?": "Test user?",
   "ifCheckedThenThisSubscriptionCanBeUsed": "If checked then this subscription can be used for previewing campaign messages",
   "erased": "[ERASED]",
+  "blacklisted": "Blacklisted",
   "confirmUnsubscription": "Confirm Unsubscription",
   "areYouSureYouWantToUnsubscribeEmail?": "Are you sure you want to unsubscribe {{email}}?",
   "unsubscribingEmail": "Unsubscribing {{email}}",
@@ -678,6 +643,8 @@
   "addSubscriber": "Add Subscriber",
   "subscribed": "Subscribed",
   "unubscribed": "Unubscribed",
+  "bounced": "Bounced",
+  "complained": "Complained",
   "parentNamespaceMustBeSelected": "Parent Namespace must be selected",
   "namespaceSaved": "Namespace saved",
   "thereHasBeenALoopDetectedInTheAssignment": "There has been a loop detected in the assignment of the parent namespace. This is most likely because someone else has changed the parent of some namespace in the meantime. Refresh your page to start anew. Please note that your changes will be lost.",
@@ -711,13 +678,11 @@
   "viewConsoleOutput": "View console output",
   "reportTemplates": "Report Templates",
   "reports": "Reports",
-  "reportName": "Report {{name}}",
-  "loadingReport": "Loading report ...",
   "outputForReportName": "Output for report {{name}}",
   "loadingReportOutput": "Loading report output ...",
-  "reportIsBeingGenerated": "Report is being generated",
+  "reportName": "Report {{name}}",
   "reportNotGenerated": "Report not generated",
-  "refresh": "Refresh",
+  "loadingReport": "Loading report ...",
   "reportName-1": "Report \"{{name}}\"",
   "output": "Output",
   "templateName": "Template \"{{name}}\"",
@@ -736,15 +701,15 @@
   "writeTheBodyOfTheJavaScriptFunctionWith": "Write the body of the JavaScript function with signature <1>function(inputs, callback)</1> that returns an object to be rendered by the Handlebars template below.",
   "useHtmlWithHandlebarsSyntaxSee": "Use HTML with Handlebars syntax. See documentation <1>here</1>.",
   "blank": "Blank",
-  "openCounts": "Open counts",
+  "allSubscribers": "All Subscribers",
-  "openCountsAsCsv": "Open counts as CSV",
+  "groupedSubscribers": "Grouped Subscribers",
-  "aggregratedOpenCounts": "Aggregrated open counts",
+  "exportListAsCsv": "Export List as CSV",
   "current": "(current)",
   "toggleNavigation": "Toggle navigation",
   "administration": "Administration",
   "users": "Users",
   "globalSettings": "Global Settings",
-  "sendConfigurations": "Send configurations",
+  "sendConfigurations": "Send Configurations",
   "logOut": "Log out",
   "home": "Home",
   "sourceOnGitHub": "Source on GitHub",
@@ -783,10 +748,6 @@
   "useast1": "US-EAST-1",
   "uswest2": "US-WEST-2",
   "euwest1": "EU-WEST-1",
-  "builtinZoneMta": "Built-in ZoneMTA",
-  "dynamicConfigurationOfDkimKeysViaZoneMt": "Dynamic configuration of DKIM keys via ZoneMTA\\'s Mailtrain plugin (use this option for builtin ZoneMTA)",
-  "dynamicConfigurationOfDkimKeysViaZoneMt-1": "Dynamic configuration of DKIM keys via ZoneMTA\\'s HTTP config plugin",
-  "noDynamicConfigurationOfDkimKeys": "No dynamic configuration of DKIM keys",
   "mailerSettings": "Mailer Settings",
   "mailerType": "Mailer type",
   "hostname": "Hostname",
@@ -808,7 +769,6 @@
   "throttling": "Throttling",
   "messagesPerHourEg1000": "Messages per hour eg. 1000",
   "maximumNumberOfMessagesToSendInAnHour": "Maximum number of messages to send in an hour. Leave empty or zero for no throttling. If your provider uses a different speed limit (messages/minute or messages/second) then convert this limit into messages/hour (1m/s => 3600m/h). This limit is per sending process.",
-  "dynamicConfiguration": "Dynamic configuration",
   "dkimSigning": "DKIM Signing",
   "zoneMtaDkimApiKey": "ZoneMTA DKIM API key",
   "secretValueKnownToZoneMtaForRequesting": "Secret value known to ZoneMTA for requesting DKIM key information. If this value was generated by the Mailtrain installation script then you can keep it as it is.",
@@ -833,9 +793,6 @@
   "trackingId": "Tracking ID",
   "uaxxxxxxx": "UA-XXXXX-XX",
   "enterGoogleAnalyticsTrackingCode": "Enter Google Analytics tracking code",
-  "googleMapsApiKey": "Google Maps API Key",
-  "xxxxxx": "XXXXXX",
-  "theMapOverviewInCampaignStatistics": "The map overview in campaign statistics requires a Google Maps API key. Please enter it here. If no key is given, Google may throttle map requests, which will result in occassional unavailability of the map in the campaign statistics.",
   "frontpageShoutOut": "Frontpage shout out",
   "htmlCodeShownInTheFrontPageHeaderSection": "HTML code shown in the front page header section",
   "gpgSigning": "GPG Signing",

setup/functions

@@ -8,6 +8,7 @@ function installBase {
     local urlBaseTrusted="$1"
     local urlBaseSandbox="$2"
     local urlBasePublic="$3"
+    local wwwHost="$4"
     
     yum -y install epel-release
     
@@ -21,7 +22,7 @@ enabled=1
 gpgkey=https://www.mongodb.org/static/pgp/server-4.0.asc
 EOT
     
-    yum -y install mariadb-server nodejs ImageMagick git python redis pwgen bind-utils gcc-c++ make mongodb-org
+    yum -y install mariadb-server nodejs ImageMagick git python redis pwgen bind-utils gcc-c++ make mongodb-org bzip2
     
     systemctl start mariadb
     systemctl enable mariadb
@@ -54,6 +55,7 @@ roUser: nobody
 roGroup: nobody
 
 www:
+  host: $wwwHost
   secret: "`pwgen -1`"
   trustedUrlBase: $urlBaseTrusted
   sandboxUrlBase: $urlBaseSandbox
@@ -67,11 +69,11 @@ redis:
   enabled: true
 
 log:
-  level: warn
+  level: info
 
 builtinZoneMTA:
   log:
-    level: info
+    level: warn
 
 queue:
   processes: 5
@@ -121,7 +123,7 @@ EOT
     
     echo "Success! Open http://$urlBaseTrusted/ and login as admin:test"
 
-    if [ -n "$firewallCmdExists" ]; then
+    if [ -z "$firewallCmdExists" ]; then
         echo "Note that firewall was not setup because firewall-cmd is missing on your system. Please make sure your firewall is correctly setup. If you are on AWS, this means to enable HTTPS and HTTP in your security group."
     fi
 }
@@ -269,7 +271,12 @@ function createCertificates {
         firewall-cmd --add-port=80/tcp
     fi
 
-    certbot certonly --agree-tos --email "${email}" --standalone -n -d "${hostPublic}" -d "${hostTrusted}" -d "${hostSandbox}"
+    certbot certonly --email "${email}" --standalone -n -d "${hostPublic}" -d "${hostTrusted}" -d "${hostSandbox}"
+
+    # Install cron
+    echo "0 3 * * * /usr/bin/certbot certonly --apache -n -d \"${hostPublic}\" -d \"${hostTrusted}\" -d \"${hostSandbox}\"" > crontab
+    crontab crontab
+    rm -rf crontab
 
     if [ -n "$firewallCmdExists" ]; then
         # Revert firewall to original state
@@ -277,3 +284,24 @@ function createCertificates {
     fi
 }
 
+
+function installService {
+    cat > /etc/systemd/system/mailtrain.service <<EOT
+[Unit]
+Description=Mailtrain server
+After=syslog.target network.target mariadb.service redis.service mongod.service
+
+[Service]
+Environment="NODE_ENV=production"
+WorkingDirectory=/opt/mailtrain/server
+ExecStart=/usr/bin/node index.js
+Type=simple
+Restart=always
+RestartSec=10
+
+[Install]
+WantedBy=multi-user.target
+EOT
+
+    systemctl daemon-reload
+}

setup/install-centos7-https.sh

@@ -25,11 +25,9 @@ Installs Mailtrain 2 on CentOS 7. This performs installation for external use. I
 a reverse HTTPS proxy using Apache HTTPD, sets up firewall rules, and obtains a certificate from Letsencrypt.
 
 You have to allocate three endpoints for Mailtrain - trusted (admin UI), sandbox (editors for templates), public (subscription forms and archive).
-These endpoints have to differ in hostname. It's fine to host them all from one IP address.
+These endpoints have to differ in hostname. It's fine to host them all from one IP address. The email parameters is needed by certbot.
 
-The email is needed by certbot. Please note that by running the script, you agree with Letsencrypt's conditions.
+Example: install-centos7-https.sh mailtrain.example.com sbox.mailtrain.example.com lists.example.com admin@example.com
-
-Example: install-centos7-https.sh mailtrain.example.com sbox.mailtrain.example.com mail.example.com admin@example.com
 EOF
 
   exit 1
@@ -49,4 +47,6 @@ createCertificates "${hostTrusted}" "${hostSandbox}" "${hostPublic}" "${email}"
 
 installHttps "${hostTrusted}" 443 "${hostSandbox}" 443 "${hostPublic}" 443 "/etc/letsencrypt/live/${hostPublic}/cert.pem" "/etc/letsencrypt/live/${hostPublic}/privkey.pem" "/etc/letsencrypt/live/${hostPublic}/chain.pem" 
 
-installBase "https://${hostTrusted}" "https://${hostSandbox}" "https://${hostPublic}" "${email}"
+installBase "https://${hostTrusted}" "https://${hostSandbox}" "https://${hostPublic}" 127.0.0.1
+
+installService

setup/install-centos7-local.sh

@@ -36,4 +36,6 @@ if [ $# -lt 0 ]; then
         HELP
 fi
 
-installBase http://localhost:3000 http://localhost:3003 http://localhost:3004
+installBase http://localhost:3000 http://localhost:3003 http://localhost:3004 0.0.0.0
+
+installService