Feature #61237
Whats new in scalix web applications .
0%
Description
This is general short or maybe not description what you can expect from Scalix web applications with next minor update ( lets say 12.8 - but there are not plans to start any other versions).
Current builds for early review you may find at http://downloads.scalix.com/upcoming/
filename will be looks like
scalix-next-GA-enterprise-redhat-suse.bin
So what changed.
1. Tomcat 9 (tomcat 8 is not mainline and may receive only some security fixes), removed unused java libraries, updated all java libraries.
2. #61050 - apply web application setting without restarting tomcat. But please note that not all settings will be reloaded , so for some settings you would need to restart tomcat. Or just wait for some time, for e.g. if you changed settings for IMAP client those settings will be applied only after user session will expire and with next access under that user new imap properties will be in imap session.
3. SWA vertical layout. #61179 vertical message pane in SWA as Office265/Outlook has.
4. #60694 SWA - Replace Javascript alerts with html pop-up dialog s
5. Generic perfermance improvements
- SWA loads 5600 emails in 20seconds.
- ActiveSync - sync to device 1600 appoinments, 53 emails (with meeting request) ~ 3min+. cpu usage average 40-50% max 99-110%.
- Less overloading IMAP . Please note that during synchronizing calendaring via DAV protocol you may see high imap cpu usage it will change as soon as DAV will be stable ...
- Postgresql cpu usage is low during filling massages cache. Before it was not high 50-90% for 3-4 seconds on saving 10k emails(not all email just some parts/data that needed for better performance). Now there are no such thing cpu usage is lower or maybe I was not able to capture it.
- Memory usage you may see free memory (top command) 120Mb(tomcat started) -> 67Mb(after some work). Please note that free memory can increased to initial 120MB after sometime due sessions for users is closing after some time of inactivity to free resources.
Tomcat memory settings is JAVA_OPTS="-server -Xms500m -Xmx500m ..." , using jre 1.8.0_131. And its enough for fill caching 6k emails in Inbox and fill 1.5k appointments.
PLEASE KEEP IN MIND despite lower memory usage it does not mean that you may dramatically decrease memory. To work normally Scalix Web application requires more or less reasonable amount of available memory. You may notice that from release to release web apps works better and may use less memory, but still for big amount of emails web apps requires not very low amount of memory.
- Its not a secret that ICAl (appointment data) is stored in Postgresql for better performance so filling ICAL data for 1.6k appointments took 3-5 seconds (checked with 12.7.1 web DAV it was ~ 13 seconds) and less cpu usage. Generally speaking now this part depends on IMAP performance. From web applications part its fast enough.
6. all web applications will use common jar library to work with XML, ICAL, MAPIXML , VCard... For 3rd party developers or clients who wants to migrate from Scalix may use this library to export Contacts -> VCARD . Or use API to get Contacts as VCARD - this is not new feature in API but now VCARD has more data. Before export scalix contact to VCARD had filled limited amount of fields.
for e.g.
<?xml version="1.0" encoding="UTF-8"?> <contact> <subject>test</subject> <direct_ref>0015458b540bcc74</direct_ref> <sensitivity>0</sensitivity> <message_class>IPM.Contact</message_class> <is_recurring>false</is_recurring> <reminder_set>false</reminder_set> <send_rich_info>false</send_rich_info> <display_name_prefix></display_name_prefix> <last_name></last_name> <email1_address>tesst@tes.com</email1_address> <suffix></suffix> <display_name></display_name> <middle_name></middle_name> <email1_address_with_comment>test <tesst@tes.com></email1_address_with_comment> <file_as>test</file_as> <first_name>test</first_name> <email1_display_name>tesst@tes.com</email1_display_name> <last_modification_time>2017-04-05T13:25:02Z</last_modification_time> <mapi_charset>UTF8</mapi_charset> </contact>
this library can transform to
# Distribution List BEGIN:VCARD VERSION:4.0 PRODID:-//Scalix Java MapiXml migrator //1.0//EN UID:001deb4da95eb1c8 KIND:group EMAIL:sdcasdas <2222@sdasdas.com> EMAIL:sdsadas <s12121@asdasdas.com> END:VCARD #Contact Item with all possible values BEGIN:VCARD VERSION:4.0 PRODID:-//Scalix Java MapiXml migrator //1.0//EN UID:001df3c9d9d908b0 N:last;first;middle;prefix;sufix FN:prefix first middle last sufix NICKNAME:nickname ANNIVERSARY:20180916 BDAY:20180916 EMAIL:"email1@site.com" <email1@site.com> EMAIL;ALTID=1:"email2@scalix.com" <email2@scalix.com> EMAIL;ALTID=2:"email3@scalix.com" <email3@scalix.com> ADR;TYPE=work:;;buisness streat;buisness stcity;buisness state;buisness zip ;buisness country ADR;TYPE=home:;;home street;home city;home state;home zip;home country ADR;TYPE=intl:;;other street;other city;other state;other ZIP;other country REV:20180910T095135Z URL:http:///asdasds.dd IMPP:IMadddd KIND:individual ROLE:profession TITLE:job ORG:COMPANY;departmen;office???/ X-MANAGER:Mangers names goe here X-ASSISTANT:asssistants names.... X-SPOUSE:spouse's name X-FILE-AS:last, first TEL;TYPE=home:home phone TEL;TYPE=work:buisness phone TEL;TYPE=fax:fax number TEL;TYPE=cell:mobile phone END:VCARD # Simpler contact info BEGIN:VCARD VERSION:4.0 PRODID:-//Scalix Java MapiXml migrator //1.0//EN UID:0015458b540bcc74 N:;test;;; EMAIL:"tesst@tes.com" <tesst@tes.com> REV:20170405T102502Z KIND:individual X-FILE-AS:test END:VCARD
7. DAV is the biggest part of API that was changed.
- Sub folders support.
- Support delegates (if you have calendar from some another scalix user in swa under "Other Users") you will be able to add this Calendar into some Dav client. So DAV now supports other user folders .
- CardDav support. Scalix Contacts will be available via CardDav protocol.
- Currently tested and suppported clients
- Thunderbird + Lightening (https://www.thunderbird.net/en-US/calendar/),
- Thunderbird + CardBook(CardDav client for Thunderbird) (https://addons.thunderbird.net/en-US/thunderbird/addon/cardbook/?src=ss),
- KOrganizer (https://userbase.kde.org/KOrganizer/en),
- Gnome Evolution (https://wiki.gnome.org/Apps/Evolution),
- eM Client (https://www.emclient.com/).
- DavDroid (dav client for Android) (https://www.davdroid.com/)
Shorter version of upcoming DAV support it was rewrriten and now use less cpu usage, less memory and works much more faster.
some stats can be outdated but
Evolution full sync 3 Calendars ~1600+ appointments started [25/Oct/2018:06:24:12 -0400] "PROPFIND /api/dav/Principals HTTP/1.1" 401 2763 .... last request [25/Oct/2018:06:25:25 -0400] "REPORT /api/dav/Calendars/sxadmin-test@user.com/0001201392fc6028/ HTTP/1.1" 207 16865 time spent : 1 minute and 13 seconds (73 seconds)
Please note that 73 seconds its because appointment data (ICAL) was cached , your results may be different due some work (fill cache data , etc)
But in general it fast.
During the day I will provide screenshots how to configure clients and will tell about client restrictions that I found.
Files
History
Updated by Alexey Bobyr over 2 years ago
- File Screenshot_20181120_080125.png Screenshot_20181120_080125.png added
- File Screenshot_20181120_080210.png Screenshot_20181120_080210.png added
- File Screenshot_20181120_080231.png Screenshot_20181120_080231.png added
- File Screenshot_20181120_080253.png Screenshot_20181120_080253.png added
- File Screenshot_20181120_080307.png Screenshot_20181120_080307.png added
- File Screenshot_20181120_080449.png Screenshot_20181120_080449.png added
- File Screenshot_20181120_080509.png Screenshot_20181120_080509.png added
- File Screenshot_20181120_080524.png Screenshot_20181120_080524.png added
Gnome Evolution (https://wiki.gnome.org/Apps/Evolution). Configuring profile.
Launch Evolution . Than go to the menu Edit -> Accounts
click on "+ ADD" button and add "Collection Account"
then
For Evolution you may use short url to API (without Principal) in my case it was
Fill password and click on "Try again"
And you will see that Evolution found Dav resources that he can support.
click "Next"
and click "Next" or "...." to finish account creation wizard.
After some time you will see folders / appoinments /contacts / tasks
Updated by Alexey Bobyr over 2 years ago
- File Screenshot_20181120_100315.png Screenshot_20181120_100315.png added
- File Screenshot_20181120_100331.png Screenshot_20181120_100331.png added
- File Screenshot_20181120_100354.png Screenshot_20181120_100354.png added
- File Screenshot_20181120_100624.png Screenshot_20181120_100624.png added
- File Screenshot_20181120_100646.png Screenshot_20181120_100646.png added
Updated API web interface for DAV protocol its responsive and has more information as had before.
The structure of Root folders changed. Tasks moved from /api/dav/Calendars/My Calendars/ to root.
Contacts its an Dav Address-book
If another user shared with you his Calendar or other folder
after accessing /api/dav/Calendars/ you will see Principals list and you must choose for which user you want see Calendar's .
/api/dav/Principals/RFC822_EMAIL/ will give you generated VCARD for selected user. Lets say 'My Vcard'.
VCARD data fills from LDAP User attributes. as for this example I fill data for sadmin in SAC
For Folders Content-type column shows what type they have . Its Appointment or Calendar folder or simple folder with emails
the same for messages listing . Content-Type column shows type of message
Updated by Alexey Bobyr over 2 years ago
- File Screenshot_20181120_104505.png Screenshot_20181120_104505.png added
- File Screenshot_20181120_104533.png Screenshot_20181120_104533.png added
- File Screenshot_20181120_104545.png Screenshot_20181120_104545.png added
- File Screenshot_20181120_104613.png Screenshot_20181120_104613.png added
- File Screenshot_20181120_104633.png Screenshot_20181120_104633.png added
- File Screenshot_20181120_104655.png Screenshot_20181120_104655.png added
- File Screenshot_20181120_104723.png Screenshot_20181120_104723.png added
- File Screenshot_20181120_104816.png Screenshot_20181120_104816.png added
- File Screenshot_20181120_105023.png Screenshot_20181120_105023.png added
- File Screenshot_20181120_105047.png Screenshot_20181120_105047.png added
KOrganizer (https://userbase.kde.org/KOrganizer/en)
Launch KOrganizer and go to the menu Settings -> Configure KOrganizer
Click on Calendaring in first group item
Click on "Add" button and Choose "Dav groupware resource"
Enter your email and password
Choose Scalix from drop down
Enter HOST or IP
Now you test if everything is ok by pressing "Test Connection" or click on finish button.
To add second Principal in dialog(account details it will appear after successful wizard ) click "Add"
In "Remote URL" enter second Principal for my me it was http://192.168.122.3/api/dav/Principals/as2@....
To check if you entered data correctly click on Fetch btn
For CardDav support please add new Protocol for KOrganizer account with type CardDav and
Remote URL http://192.168.122.3/api/dav/Contacts
Close all dialogs .
And you will be able to see folders appointments etc.
Please note that CARDDAv in KOrganizer shows contacts list only when you are adding Attendee for an appointment
Updated by Alexey Bobyr over 2 years ago
Updated by Alexey Bobyr over 2 years ago
- File Screenshot_20181120_111933.png Screenshot_20181120_111933.png added
- File Screenshot_20181120_112016.png Screenshot_20181120_112016.png added
- File Screenshot_20181120_112104.png Screenshot_20181120_112104.png added
- File Screenshot_20181120_112124.png Screenshot_20181120_112124.png added
- File Screenshot_20181120_112201.png Screenshot_20181120_112201.png added
Adding Calendars to the ThunderBird + Lightening (https://www.thunderbird.net/en-US/calendar/) , the same as it was.
This Client does not support DAv Groupware.
as for Thunderbird + CardBook(CardDav client for Thunderbird) (https://addons.thunderbird.net/en-US/thunderbird/addon/cardbook/?src=ss) its the same you must specify direct url to the Folder with Contacts.
But this clients has option to locate all folders during wizard dialog .
Click on menu Tools -> CardBook
right click somewhere in left panel and in menu click "New Address Book"
Choose "Remote" and click "next" btn
In URL field enter Principal collection of Contact folders . For me url is https://192.168.122.3/api/dav/Contacts/sxadmin-test@..../
and enter credentials and click on "Validate" button . If everything will be ok "Next" button will be clickable.
And you will list of folders which contains Contacts. You may choose what folder to add and which not needed , also as you can see you can set ReadOnly mode if needed
After all steps in CardBook wizard all folders that you choose will be in left pane.
Updated by Alexey Bobyr over 2 years ago
- File Screenshot_20181120_113431.png Screenshot_20181120_113431.png added
- File Screenshot_20181120_113756.png Screenshot_20181120_113756.png added
- File Screenshot_20181120_114807.png Screenshot_20181120_114807.png added
- File Screenshot_20181120_114823.png Screenshot_20181120_114823.png added
- File Screenshot_20181120_114834.png Screenshot_20181120_114834.png added
eM Client (https://www.emclient.com/)
Launch EM Client. If you does not have account you will see account setup wizard , and if you want to add new account please go to the "Menu" -> "Tools" -> "Accounts" and click in "+" button.
Click on "Calendar Group" and click on "CalDav" icon(button)
Fill data. Principal url and credentials to access . In my case it was https://192.168.122.3/api/dav/Principals/sxadmin-test@...../
Finish wizard and close all dialog and after that you can choose what Folder sync ...
Please note that eM Client can handle http://IP_OR_HOST/api/dav and there is no need to specify concrete Principal
and have several accounts. But when I have tried to use http://IP_OR_HOST/api/dav for an account its working as in Gnome Evolution(all folders for all principals client shows) until you close em Client.
If you close eM Client you want see folders fetched throw DAV protocol.
Does not know if it something at our side or some issue at client side or its because I'm running eM client using wine ("Wine Is Not an Emulator", application which allows run windows applications in linux OS.)
So if somebody has emClient and can check behavior for url http://IP_OR_HOST/api/dav at windows machine I would appreciate for the help.
Updated by Alexey Bobyr over 2 years ago
- File Screenshot_20181120_121730.png Screenshot_20181120_121730.png added
- File Screenshot_20181120_121740.png Screenshot_20181120_121740.png added
- File Screenshot_20181120_121807.png Screenshot_20181120_121807.png added
- File Screenshot_20181120_121824.png Screenshot_20181120_121824.png added
- File Screenshot_20181120_122009.png Screenshot_20181120_122009.png added
DavDroid (dav client for Android) (https://www.davdroid.com/)
At Android VM I used F-Droid application to install DavDroid you may install it using Play store.
For Tasks support installed "OPenTasks" application using F-Droid application . But you can install DavDroid via Play Store and when you open this application for first time it should ask you to Install "Tasks" application or do not show this message or ...
Launch DavDroid application and press "+" (add account)
Select "Login with url and user name"
URL : https://192.168.122.3/api/dav/
username: your user
pswd : user password
and press to the "Login"
Choose account display name and press "Create account"
After that press on newly create account in account list.
Tick folders which you want to sync. and Press to the sync icon .
and after some time you will see appointments/contacts and tasks at your device fetch via DAV protocol
Updated by Alexey Bobyr over 2 years ago
Well now what is missing at this moment.
1. CardDav read only. VCARD from dav clients must be transformed to scalix format. Maybe SWA requires some additional functional to save/update contacts properly.
2. Calendar schedule inbox / outbox does not work at this moment. Its appointments that you received but have not accept them or declined. And 'outbox' its appointments that you sent to recipients but have not received reply from attendee .
3. Contacts will have special Folders "System addressbook" and "Personal addressbook" like SWA has. with ability to add custom LDAP server and fetch contacts from non scalix LDAP server. So SWA addressbook functionality will be available for DAV AddressBook.
4. DAV ACL is not implemented. for previous versions DAV ACL was in initial state. So most probably it will be the feature.
Restrictions:
1. Old DAV urls may not work. So when customer will upgrade to that version he must inform all users to recreate calendar items in Thunderbird + Lightening (for this client its critical). For clients that use DavGroupWare folders will resync automatically at least they must.
2. To rename public folder in DAV client which looks like "Calendars (Public Folder)" user must remove '(Public Folder)' and then edit 'Calendars'. (Public Folder) added automatically. the same for subfolders "Calendars\subfolder2" need to remove "Calendars\" because it added automatically. For other user folders "Calendar (other user display name)" need remove "(other user display name)".
Adding/Updating/deleting appointments works normally for current implemention.
It seems that's all for now . Maybe I miss something but will update this ticket if it would be needed.
Updated by Alexey Bobyr over 2 years ago
New configurable options for DAV implementation is here #61239 .
Updated by Alexey Bobyr over 2 years ago
Sorry forgot to include DavDroid Contacts screenshot
Updated by Alexey Bobyr over 2 years ago
To support Tasks with DavDroid .
Please install OpenTasks before DavDroid or if you already have DavDroid than install OpenTasks(Tasks) (in F-Droid, at Play Store name can be different have not checked) and reinstall DavDroid . Please note that you have to create account for scalix DAV again.
And it will work.
Updated by Rene Blechschmidt over 2 years ago
Hi Alexey ,
Can you remove me from this Task. Thx for this information, but thats too much information ;-)
Updated by Alexey Bobyr over 2 years ago
Similar to SWA functionality . Scalix CardDav will have System Directory and Personal Directory as folders
Please note that this functionality may impact to LDAP server performance and will cause more memory usage at API side due cashing LDAP results. New configuration options for API. You can disable it if you want.
# # Display "Scalix System Directory" and "Scalix Personal Directory" virtual folders in list of available # Contacts folders. Similar to SWA functionality "Scalix System Directory" will show all users that are in System # directory fetched from LDAP server, "Scalix Personal Directory" will show all users that are in Personal Directory # fetched from LDAP server. # platform.dav.ldap.addressbook.enabled=true # # Amount of time (in minutes) for how long we should keep LDAP entry results for CardDav 'System Directory' # and 'Personal Directory' virtual folders. # platform.dav.ldap.addressbook.cache_timout=10
Updated by Prakash Nikhar about 2 years ago
Hi,
checked point 1 to 7 (some points are remaining to test in point no.7) observation as following:
1 Tested: OK
[root@sx1280 ~]# rpm -qa | grep scalix-tomcat scalix-tomcat-9.0.14-56.noarch scalix-tomcat-connector-12.6.0-14721.rhel7.noarch [root@sx1280 ~]# java -version java version "1.8.0_152" Java(TM) SE Runtime Environment (build 1.8.0_152-b16) Java HotSpot(TM) Client VM (build 25.152-b16, mixed mode, sharing) [root@sx1280 ~]#
2 # 61050 Tested: OK
3 # 61179, Tested: OK, but can we use word "Reading Pane at Bottom" instead "Horizontal Split" and "Reading Pane on the Right" instead of "Vertical split" And can add "Reading Pane Off" option if user do want see preview of mail.
4 #60694tested: Ok
5 Tested: OK
swa loads 5600 emails in 25 seconds
Active sync- 1600 appointments and 60 email meeting request ~ 5min+. cpu 45 to 60 %
Memory- free memory(tomact started) 611MB out of 1822MB no user logon, after 4 user logon 377MB free
and when all user session closed free memory increased to 981MB
my tomcat memory setting is JAVA_OPTS="-server -Xms512m -Xmx1024m -Dfile.encoding=UTF-8 -Djava.net.preferIPv4Stack=true"
and i using jre 1.8.0_152
6 Tested: OK
Contact, VCARD has more data comapare to earlier version.
API: http://192.168.43.75/api/dav/Contacts/swatest1@co75test.com/00010c02bcb9d59c/0001b202caad9712.vcf
BEGIN:VCARD VERSION:4.0 PRODID:-//Scalix Java MapiXml migrator //1.0//EN UID:0001b202caad9712 N:1;Contact;Test;Mr;Engg FN:Mr Contact Test 1 Engg NICKNAME:Engg ANNIVERSARY:20140501 BDAY:19800101 EMAIL:contact.test1@abc.com EMAIL;ALTID=1:contact.test1@pqr.com EMAIL;ALTID=2:contact.test1@xyz.com ADR;TYPE=work:;;AK Road;Mumbai;MAHARASHTRA;400000;India ADR;TYPE=home:;;KS Road;Thane;Maharashtra;421204;India ADR;TYPE=intl:;;PD Road;Mumbai;Maharashtra;400072;India REV:20190123T125010Z URL:www.test.com IMPP:contact.test KIND:individual ROLE:IT TITLE:Engg ORG:ABC LTD;IT;XYZ X-MANAGER:ABC X-ASSISTANT:PQR X-SPOUSE:XYZ X-FILE-AS:Contact 1 TEL;TYPE=home:8765432109 TEL;TYPE=work:9876543210 TEL;TYPE=fax:7654321098 TEL;TYPE=cell:6543210987 END:VCARD
7) DAV
Subfolder: working
Thunderbird + Lightening: found issue in lightning configuration, issue reported in #61266
Note: some test are remaining for point no. 7