As I said in my last blog, the Alfresco Summit 2014 took place in London from October 7 to 9. So yeah, unfortunately yesterday was the last day of this amazing event. It’s the first time I had the chance to participate in an Alfresco Summit but I’m quite sure it will not be the last! I can’t tell what moment I preferred during this event because the Keynote (presentation of Alfresco One 5.0) was very impressive and I really enjoyed it! All presentations were very interesting too but if I should pick just one thing, I think I would choose… The two evening parties :-D.

So more seriously, on the second day of conferences, I had the opportunity to attend a lot of sessions about best practices. The principal reason why I choose these kind of session is because I wanted to confront dbi services’ best practices about Alfresco to best practices of other Alfresco consultants (either Alfresco employees or Alfresco partners). Best practices sessions covered several different fields like:

• Security
• Performance
• Replication, Backup & Disaster Recovery
• Monitoring
• Sizing
• Aso…

Yesterday morning, I had the chance to attend to a session presented by Miguel Rodriguez (Technical Account Manager at Alfresco) about how to monitor an Alfresco installation. For the end of this blog, I will summarize the presentation of Miguel to show you how this solution is powerful! So the presentation wasn’t just about how to monitor a JVM, it was really more advanced because Miguel presented how to control all Alfresco environments (Dev, Test, Prod, aso…) in a single place with several tools.

The monitoring of Alfresco can be divided into three different categories:

• Monitoring and treatment of Alfresco logs
• Monitoring of the OS/JVM/Alfresco/Solr/Database
• Sending alerts to a group of people regarding the two categories above

I. Monitor and treat Alfresco logs

This first part about the monitoring and treatment of Alfresco logs needs three software:

• Logstash: it’s an utility that can be used to monitor events or log files and to execute little commands (like a “top”)
• ElasticSearch: this tool is able to index log files to do some restful search and analytics after that. This will be very useful for this part
• Kibana3: the last software is the one used for the interface. It will just be used to query the index created by ElasticSearch and beautifully display the results. The Kibana3 page is composed of boxes fully customizable directly from the interface and that are refreshed automatically every X seconds (also customizable)

So here, we have a complete log monitoring & treatment solution. This solution can be used to display all errors in all alfresco log files in the last X secondes/minutes/hours, aso… But it can also be used to apply filters on all log files. For example, if you want to find all log entries related to “License”, then you just have to type “license” in the search box. As all log entries are indexed by ElasticSearch (using Lucene), then all boxes on the interface will be refreshed to only display data related to “license”.

Image from Jeff Potts’ blog. This image display data related to JMeter but same results could be applied to Alfresco.

II. Monitoring of the OS/JVM/Alfresco/Solr/Database/…

This second part is as simple as the first one:

• Logstash: this time, logstash is more used to execute commands to retrieve numerical data. These commands can be Unix commands (e.g. “top” to retrieve current CPU consumption) but they can also be JMX commands to retrieve certain values from the JVM like free heap space. More details are available below.
• Graphite: this utility will just retrieve all numerical information generated by Logstash and then store them in a database
• Grafana: as above, this part need an interface… And in the same way, the page is fully customizable directly from the interface. You can add as many boxes (each box represent a diagram, often a line chart) as you want with as many information available in your database.

When I first saw the interface of Grafana, I thought “Oh, it looks like jconsole”. But this was really only my first impression. On this interface, you can add different boxes to display numerical data about not only one Alfresco Node but about all Alfresco Nodes! For example, you will be able to display boxes with the CPU consumption, memory consumption, the JVM heap size (permgen, used, free, aso…) and all these boxes will show data related to all Nodes of your cluster. All this kind of stuff is already available through jconsole. But in addition to what jconsole is capable of, you will be able to display boxes related to (non-exhaustive list):

• All database sessions available and all used
• Tomcat threads
• Disk i/o
• Alfresco documents in the repository and in Solr (Solr store documents in a different way)
• Health of the indexing: based on different values, we can know if the indexing is working well or if some tuning is required
• Index workers and tracking status: this chart show if there is a gap between the tracking of documents by Solr and the indexing of these documents which would means that Solr isn’t indexing at that time (probably an issue)
• Solr transactions
• Concurrent users
• Aso…

 Image extracted from the demonstration of Miguel Rodriguez

III. Sending alerts

You are probably aware of some tools like Nagios that can be used to send alerts of different sorts. So let me present you a tool that you may not know: Icinga. This is the last tool that we will use for this monitoring solution and it’s pretty much the same thing that Nagios (it’s a fork of Nagios (2009)). With this tool, you are able to define some rules with a low threshold and a high threshold. If a threshold is reached, then the status of the rule is updated to Warning or Critical (e.g. Server Free Memory below 20% = Warning ; Server Free Memory below 10% = Critical). At the same time, an alert is sent to the group of people defined as contact in case of threshold reached. Well it’s pretty much the same thing that Nagios or other tools like that so I will not describe Icinga deeper.

 Image from Toni de la Fuente’s Blog

So here we are, we now have a lot of different tools used to provide a really good passive and active monitoring solution. But I hear your complaints from here about “Hey, there are way too much components to install, this is too difficult!”. Well let me finish! All tools above are Open Source tools and are available as a package created by Packer. This utility let you create a virtual machine “template” with all needed components pre-installed. So you are able to just download the package and after the execution of a single command, you will have a new virtual machine up and running with all monitoring components. The only remaining step is to install a Logstash agent on each components of Alfresco (Alfresco Nodes, Solr Nodes) and configure this agent to retrieve useful data.

I was really impressed by this session because Miguel Rodriguez showed us a lot of very interesting tools that I didn’t know. His monitoring solution looks really pretty well and I’m quite sure I will take some time to try it in the next few weeks.

I hope you have enjoyed this article as much as I enjoyed Miguel’s session and if you want to download your own version of this Monitoring VM, take a look at Miguel’s Github account.

Cet article Alfresco Summit 2014 – Conferences, Day 2 – Best Practices & Monitoring est apparu en premier sur Blog dbi services.

I didn’t have the opportunity to post a new entry about Alfresco in this blog for a long time now, so I will fix this! In this blog entry, I will talk about a bug I encountered a few months ago. I resolved it but I, so far, not had the time to share my knowledge with you.

I. Description of the issue

This bug appears no matter what the version of Alfresco is used, regardless of the components that are installed, aso… So what is this bug? In fact, this bug isn’t blocking anything. Actually it has no impact on the daily work, however, it fills up the Alfresco log files very quickly which can be problematic if you are an administrator searching for information in these log files! Indeed, each time a user accesses a page of Alfresco, between 10 and 50 Java Exceptions are generated (always the same), this create gigabytes log files in minutes/hours. Here is the exception I’m talking about:

...
Jul 08, 2014 10:42:16 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 95898 ms
2013-07-08 10:45:02,300 INFO [web.site.EditionInterceptor] [http-apr-8080-exec-1] Successfully retrieved license information from Alfresco.
2013-07-08 10:45:02,417 ERROR [extensions.webscripts.AbstractRuntime] [http-apr-8080-exec-3] Exception from executeScript - redirecting to status template error: 06080001 Unknown method specified to remote store API: has
org.springframework.extensions.webscripts.WebScriptException: 06080001 Unknown method specified to remote store API: has
at org.alfresco.repo.web.scripts.bean.BaseRemoteStore.execute(BaseRemoteStore.java:326)
at org.alfresco.repo.web.scripts.RepositoryContainer$3.execute(RepositoryContainer.java:426)
at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:433)
at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecute(RepositoryContainer.java:495)
at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecuteAs(RepositoryContainer.java:533)
at org.alfresco.repo.web.scripts.RepositoryContainer.executeScript(RepositoryContainer.java:276)
at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:377)
at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:209)
at org.springframework.extensions.webscripts.servlet.WebScriptServlet.service(WebScriptServlet.java:118)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.alfresco.web.app.servlet.GlobalLocalizationFilter.doFilter(GlobalLocalizationFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:929)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1002)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:585)
at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1813)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
...


The first time I encountered this exception, it was on an Alfresco v4.x installation, up and running for some years with a lot of extensions/customizations (mostly .AMP files). If you need more information about AMPs, it means Alfresco Module Package, it’s the better way to extend Alfresco. Please take a look at some of my old blogs to find information about how to create this kind of stuff!

I had this exception on more than one Alfresco server, and because of that, I firstly thought that this exception came from an AMP… Therefore, I went through all these extensions but despite hours of research, I found nothing.

II. How to replicate the issue

I tried to replicate this issue with a fresh installation of Alfresco, same version, same extensions, aso… But I haven’t been able to do so, at the first place. Finally, one day, I found out that there were something strange with the Alfresco servers on which the Java Exceptions appeared: the “Sites” folder weren’t there. Indeed, after the installation of a new Alfresco server, the deletion of the default site (“Sample: Web Site Design Project”) and the deletion of the “Sites” folder from the Repository browser, the exception appeared magically in the Alfresco log files… Now that we know from where this issue comes from, it’s quite easy to replicate it:

1. Install a new Alfresco server with the same version from the bundle executable/binaries (quicker)
2. Start the Alfresco server and open the Alfresco Share UI (http://localhost:8080/share) using the admin account
3. Navigate to the Sites finder (http://localhost:8080/share/page/site-finder)
4. Click on “Search” to display all existing sites (only the default one is present: “Sample: Web Site Design Project”)
5. Click on “Delete” to delete the swsdp site
6. Navigate to the Repository (http://localhost:8080/share/page/repository)
7. Remove the “Sites” folder on the Repository page (/Company Home/Sites)
8. Refresh the page and take a look at your logs

After doing that, you should be able to see a lot of exceptions like the one describe above. Issue replicated!

III. How to solve the issue

Being able to replicate an issue is good, but knowing how to solve it is better!

If the “Sites” folder has been deleted in the first place, it was because the Alfresco Sites weren’t used at all. Therefore, the simplest solution to resolve this issue was to get the “Sites” folder back. But it’s not that easy because this folder has a particular type, some particular attributes, aso… You can’t just create a new folder, rename it “Sites” and hope that it will work ;). Starting from here, what you can do to solve this issue is:

1. Restore the “Sites” folder using a backup
2. Replicate the “Sites” folder from another Alfresco server

If you don’t have a way to restore the “Sites” folder like it was my case (after some months, no backup left), here is what you can do to fix this issue:

Let’s say that the Alfresco server, where the “Sites” folder doesn’t exist anymore, is named “A”. Please take a look at the end of this blog entry for some screenshots that may help you.

1. Install a new Alfresco server with the same version as “A” from the bundle executable/binaries. This can be on your local machine. Let’s name this Alfresco server “B”
2. Start the Alfresco server “B” and open the Alfresco Share UI (http://localhost:8080/share) using the admin account
3. Navigate to the Sites finder (http://localhost:8080/share/page/site-finder)
4. Click on “Search” to display all existing sites (only the default one is present: “Sample: Web Site Design Project”)
5. Click on “Delete” to delete the swsdp site
6. Navigate to the Repository (http://localhost:8080/share/page/repository) (DON’T delete the “Sites” folder)
7. Configure a replication target on “B” to point to “A” (take a look at the Alfresco doc: http://docs.alfresco.com/4.1/tasks/adminconsole-replication-transfertarget.html)
8. Enable the replication:
   1. Add the Alfresco Share url and the RepositoryId of “A” into the share-config-custom.xml file of “B” (take a look at the Alfresco doc: http://docs.alfresco.com/4.1/tasks/adminconsole-replication-lockedcontent.html)
   2. Add the “replication.enabled=true” into the alfresco-global.properties file of “B” (take a look at the Alfresco doc: http://docs.alfresco.com/4.1/tasks/replication-share.html)
   3. Restart “B” for the changes to be taken into account by Alfresco
9. Configure a replication job on “B” to replicate the “Sites” folder from “B” to “A” (http://localhost:8080/share/page/console/admin-console/replication-jobs)
10. Run the replication job on “B”
    

Configure the Replication Target on B (step 7 – create a folder named “TransfertToA” and edit its permissions):

Find the Repository ID of A (step 8.1):

Configure the share-config-custom.xml file of B (step 8.1):

Once the replication job has run on “B”, the exceptions will disappear from the log files of “A”. I didn’t go deeper so I don’t really know if you can create new sites using this newly imported “Sites” folder but if you removed this folder in the first place, I would guess that you don’t really need it ;).

Thank you for reading this post and I hope this will help. If you need more information, don’t hesitate to let a little comment below. See you soon for more blogs!

Cet article Exception from executeScript in Alfresco Share est apparu en premier sur Blog dbi services.

I guess you all already know what a lifecycle is. We born, we live and we die… In fact, it’s exactly the same for Aflresco Nodes! Well, at least from an end user point of view. But what is really going on behind that? This is what I will try to explain in this post.

First of all what is an Alfresco Node? For most people, a Node is just a document stored in Alfresco but in reality it’s much more than that: everything in Alfresco is a Node! A Node has a type that defines its properties and it also have some associations with other Nodes. This is a very short and simplified description but we don’t need to understand what exactly a Node is to understand the lifecycle process. So as said above, Alfresco Nodes have their own lifecycle but it’s a little bit more complicated than just three simple steps.

Please note that in this post, I will use $ALF_HOME as a reference to the location where alfresco has been installed (e.g. /opt/alfresco-4.2.c) and $ALF_DATA as a reference to the alf_data location. By default the alf_data folder is $ALF_HOME/alf_data/.

I. Creation

In this post I will use a document as an Alfresco Node to easily understand the process. So this lifecycle start with the creation of a new document named “Test_Lifecycle.docx” with the following creation date: Febrary the 1st,2015 at 16:45.

When a document is created in Alfresco, three things are done:

• File System: the content of this file is stored on the Alfresco “Content Store”. The Content Store is by default under $ALF_DATA/contentstore. This file is actually put somewhere under this folder that depends on the creation time and an ID is given to this file. For our file, it would be: $ALF_DATA/contentstore/2015/2/1/16/45/408a6980-237e-4315-88cd-6955053787c3.bin.
•  Database: the medatada of this file are stored on the Alfresco Database. In fact in the DB, this document is mainly referenced using its NodeRef or NodeID. This NodeRef is something we can see on the Alfresco Web Interface from the document details page (web preview of a document): http://HOSTNAME:PORT/share/page/document-details?nodeRef=workspace://SpacesStore/09a8bd9f-0246-47a8-9701-29436c7d29a6. Please be aware that the NodeRef contains an UUID but it’s not the same that the ID on the Content Store side… Moreover, there is a property in the DB that link the NodeRef to the Content Store’s ID for Alfresco to be able to retrieve the content of a file.
• Index: an index is created for this file in the Search engine (can be Lucene for older versions of Alfresco or Solr for newer versions). This index is in the “workspace” store.

II. Update, Review, Approve, Publish, aso…

Once the document is created, his life really begins. You can update/review/approve/publish it manually or automatically with different processes. All these actions are part of the active life of a document. From an administration point of view, there isn’t that much to say here.

III. Deletion – User level

When a document isn’t needed anymore, for any reason, a user with sufficient permissions is able to delete it. For our example, let’s say that a user deleted our file “Test_Lifecycle.docx” using the Alfresco Share Web Interface on Febrary the 20th, 2015 at 15:30 (19 days after creation). When using the Web Interface or Web Services to delete a document, the “nodeService.deleteNode” method is called. So what happened to our “three things”?

• FS: nothing changed on the Content Store. The file content is still here.
• DB: on the DB side, the NodeRef changed from workspace://SpacesStore/09a8bd9f-0246-47a8-9701-29436c7d29a6 to archive://SpacesStore/09a8bd9f-0246-47a8-9701-29436c7d29a6 (the “store_id” field changed on the “alf_node” table).
• Index: same thing for the search index: the index is moved from the “workspace” store to the “archive” store.

Actually, when a user deletes a document from a Web Interface of Alfresco, the document is just moved to a “global trashcan”. By default all users have access to this global trashcan in Alfresco Explorer to restore the documents they may have deleted by mistake. Of course they can’t see all documents but only the ones related to them. On Alfresco Share, the access to this global trashcan is configured in the share-config.xml file and by default on most versions, only administrators have access to it.

The only way to avoid this global trashcan is to programmatically delete the document by applying the aspect “cm:temporary” to the document and then call the “nodeService.deleteNode” on it. In that way, the document is removed from the UI and isn’t put in the global trashcan.

 IV. Deletion – Administration level

What I describe here as an “Administration level” is the second level of deletion that happen by default. This level is the deletion of the document from the global trashcan. If the document is still in the global trashcan, Administrators (or users if you are using Alfresco Explorer) can still restore the document. If the document is “un-deleted”, then it will return exactly where it was before and of course metadata & index of this document will be moved from the “archive” store to the “workspace” store to return in an active life.

On April the 1st, 2015 at 08:05 (40 days after deletion at user level), an administrator decides to remove “Test_Lifecycle.docx” from the global trashcan. This can be done manually or programmatically. Moreover, there are also some existing add-ons that can be configured to automatically delete elements in the trashcan older than XX days. This time, the “NodeArchiveService.purgeArchiveNode” method is called (archiveService.purge in some older Alfresco versions). So what happened to our “three things” this time?

• FS: still nothing changed on the Content Store. The file content is still here.
• DB: on the DB side, the document is still there but some references/fields (not all) are removed. All references on the “alf_content_data” table are removed when only some fields are emptied on the “alf_node” table. For Alfresco 4.0 and below, the “node_deleted” field on the table “alf_node” is changed from 0 to 1. On newer versions of Alfresco, the “node_deleted” doesn’t exist anymore but the QNAME of the node (field “type_qname_id”) on the “alf_node” table is changed from 51 (“content”) to 140 (“deleted”). So the Node is now deleted from the global trashcan and Alfresco knows that this node can now be safely deleted but this will not be done now… Once the “node_deleted” or “type_qname_id” is set, the “orphan_time” field on the “alf_content_url” table for this document is also changed from NULL to the current unix timestamp (+ gmt offset). In our case it will be orphan_time=1427875500540.
• Index: the search index for this Node is removed.

As you can see, there are still some remaining elements on the File System and in the DB. That’s why there is a last step in our lifecycle…

V. Deletion – One more step.

As you saw before, the document “Test_Lifecycle.docx” is now considered as an “orphaned” Node. On Alfresco, by default, all orphaned Nodes are protected for 14 days. That means that during this period, the orphaned Nodes will NOT be touched at all. Of course this value can be changed easily on Alfresco configuration files. So what happen after 14 days? Well in fact every day at 4am (again, by default… can be changed), a scheduled job (the “contentStoreCleaner”) scan Alfresco for orphaned Nodes older than 14 days. Therefore on April the 15th, 2015 at 04:00, the scheduled job runs and here is what it does:

• FS: the content file is moved from $ALF_DATA/contentstore/ to $ALF_DATA/contentstore.deleted/
• DB: on the DB side, the document is still here but the line related to this document on the “alf_content_url” table (this table contains the orphan_time and reference to the FS location) is removed.
• Index: nothing to do, the search index was already removed.

You can avoid this step where documents are put on the “contentstore.deleted” folder by setting “system.content.eagerOrphanCleanup=true” in the alfresco-global.properties configuration file. If you do so, after 14 days, the document on the File System is not moved but will be deleted instead.

VI. Deletion – Still one more step…!

As said before, there are still some references to the “Test_Lifecycle.docx” document on the Alfresco Database (especially on the “alf_node” table). Another scheduled job, the nodeServiceCleanup runs every day at 21:00 to clean everything that is related to Nodes that has been deleted (orphaned nodes) for more than 30 days. So here is the result:

• FS: the content file is still on the $ALF_DATA/contentstore.deleted/ folder
• DB: the DB is finally clean!
• Index: nothing to do, the search index was already removed.

VII. Deletion – Oh you must be kidding me!?

So many steps, isn’t it! As saw before, the only remaining thing to do is to remove the content file from the $AL_DATA/contentstore.deleted/ folder. You probably think that there is also a job that do that for you after XX days but it’s not the case, there is nothing in Alfresco that deletes the content file from this location. In consequences, if you want to clean the File System, you will have to do it by yourself.

On Unix for example, you can simply create a crontab entry:

50 23 * * * $ALF_HOME/scripts/cleanContentStoreDeleted.sh
Then create this file with the following content:

#!/bin/sh
CS_DELETED=$ALF_DATA/contentstore.deleted/
# Remove all files from contentstore.deleted older than 30 days
find $CS_DELETED -type f -mtime +30 | xargs rm 2> /dev/null

# Remove all empty folders from contentstore.deleted older than 60 days
find $CS_DELETED -type d -mtime +60 -empty | xargs rm -r 2> /dev/null

Please be aware that you should be sure that the folder “$ALF_DATA/contentstore.deleted” exist… And when I say that, I mean YOU MUST ABSOLUTELY BE SURE that it exists. Please also never remove anything under the “contentstore” folder and never remove the “contentstore.deleted” folder itself!

You may wonder why the DB isn’t cleaned automatically when the trashcan is cleaned and why the file content is also kept 14 days… Well I can assure you that there are several reasons and the principal one is for backup/restore performance concerns. I will not explain it in details but basically as the file content isn’t touched for 14 days by default, that means that you can restore your database up to 14 days in the past and your database will still be consistent with the File System without to restore the FS! Of course if you just do that you will lose the documents uploaded/changed in the last 14 days because your database wasn’t aware of these files 14 days ago. But you can just backup/restore the content files created in the last 14 days with an incremental backup.

E.g.: Today (05-May-2015), I want to backup the FS (only the last 14 days), then I will have to backup all folders inside $ALF_DATA/contentstore/2015/5 AND I will also have to backup all folders inside $ALF_DATA/contentstore/2015/4 with a folder name bigger or equal than 30 (days in Apr) + 5 (days in May) – 14= 21.
I hope this post was clear enough because it’s true that it can be hard to understand everything regarding the lifecycle of Alfresco Node and to deal with it properly.

Cet article Understand the Lifecycle of Alfresco Nodes est apparu en premier sur Blog dbi services.

In my previous post, I talked about the Lifecycle of Alfresco Nodes. You may have noticed that I tried to insert in my explanations some elements that are specific to databases (tables, fields, aso…). These elements are quite essential to prepare a post like this one: more database oriented. I already explained what exactly are the consequences on the database side when a node is removed and I will try in this post to share some useful queries regarding these points but not only!
For this post, I used my local Alfresco Community 4.2.c installation with a PostgreSQL database. For your information, it just take 30 minutes to get this test environment ready with the Alfresco’s installer (Windows, Mac or Unix). Of course, use the Database only for your daily administration work is certainly not the best idea but in some cases, it can really be faster and easier to just run some SQL commands at the DB level…

I. Document information

So let start this post with some generic queries that can be used to retrieve some information about documents. In this part, all columns of the results will be the same because I just pick up the same fields in my queries but the filter part (the WHERE clause) changes a little bit to be able to retrieve some information from different elements.

The first command I would like to show you is how to retrieve some information about documents based on the size of the content. Here, I just uploaded the document “Test_Lifecycle.docx” with a size of 52MB. So based on that, let’s say that I want to retrieve all elements on my Alfresco installation with a content that is bigger than40MB. In the same approach, you can select all elements with a content that is smaller than or between XX and YYMB. The conversion in MB is done using the round() function. Therefore, if you want this value to be in KB instead, just remove one division by 1024 in each round() function:

All documents bigger than 40MB

SELECT n.id AS "Node ID",
n.store_id AS "Store ID",
round(u.content_size/1024/1024,2) AS "Size (MB)",
n.uuid AS "Document ID (UUID)",
n.audit_creator AS "Creator",
n.audit_created AS "Creation Date",
n.audit_modifier AS "Modifier",
n.audit_modified AS "Modification Date",
p1.string_value AS "Document Name",
u.content_url AS "Location"
FROM alf_node AS n,
alf_node_properties AS p,
alf_node_properties AS p1,
alf_namespace AS ns,
alf_qname AS q,
alf_content_data AS d,
alf_content_url AS u
WHERE n.id=p.node_id
AND ns.id=q.ns_id
AND p.qname_id=q.id
AND p.long_value=d.id
AND d.content_url_id=u.id
AND p1.node_id=n.id
AND p1.qname_id IN (SELECT id FROM alf_qname WHERE local_name='name')
AND round(u.content_size/1024/1024,2)>40
ORDER BY u.content_size DESC;

I will just put it once but here is the result of this command in this case:

 Node ID | Store ID | Size (MB) |          Document ID (UUID)          | Creator |         Creation Date         | Modifier |       Modification Date       |    Document Name    |                            Location                            
---------+----------+-----------+--------------------------------------+---------+-------------------------------+----------+-------------------------------+---------------------+----------------------------------------------------------------
  131856 |        6 |     52.00 | eb267742-c018-4ba5-8ca4-75ca23c860f0 | Morgan  | 2015-04-30T12:05:50.613+02:00 | Morgan   | 2015-04-30T12:05:50.613+02:00 | Test_Lifecycle.docx | store://2015/4/30/12/5/0e111f05-7fcf-4a44-b719-b94cd04dd5ab.bin

So why did I selected these fields?!

• Node ID: can be useful to join different tables
• Store ID: a Store ID of 6 means that your document is in its active life. A Store ID of 5 means that this document has been deleted by a user and is now in the global trashcan
• Size (MB): what we are searching for…
• Document ID (UUID): the unique identifier of this document. The simplest way to preview this document is just to open the following url in any browser: http://HOSTNAME:PORT/share/page/document-details?nodeRef=workspace://SpacesStore/eb267742-c018-4ba5-8ca4-75ca23c860f0 (workspace://SpacesStore for store_id=6)
• Creator, Modifier, Dates: well…
• Document Name: can be useful to know the type of document without opening an URL (file extension)
• Location: the actual location of the content’s file on the File System. The “store://” refers to $ALF_DATA/contentstore/

The second command I would like to show you is how to retrieve some information based on the actual UUID of a document. As explained above, the UUID of a document can be found in the URL of its detail’s page:

A document using its UUID

SELECT n.id AS "Node ID",
n.store_id AS "Store ID",
round(u.content_size/1024/1024,2) AS "Size (MB)",
n.uuid AS "Document ID (UUID)",
n.audit_creator AS "Creator",
n.audit_created AS "Creation Date",
n.audit_modifier AS "Modifier",
n.audit_modified AS "Modification Date",
p1.string_value AS "Document Name",
u.content_url AS "Location"
FROM alf_node AS n,
alf_node_properties AS p,
alf_node_properties AS p1,
alf_namespace AS ns,
alf_qname AS q,
alf_content_data AS d,
alf_content_url AS u
WHERE n.id=p.node_id
AND ns.id=q.ns_id
AND p.qname_id=q.id
AND p.long_value=d.id
AND d.content_url_id=u.id
AND p1.node_id=n.id
AND p1.qname_id IN (SELECT id FROM alf_qname WHERE local_name='name')
AND n.uuid='eb267742-c018-4ba5-8ca4-75ca23c860f0';

Another possible command would be to find some information based on the File System location. That can be useful for example if there is a big document on the File System and you want to know the type of this document with the extension, its name or maybe some other information about the creator/modifier:

A document using its path on the File System

SELECT n.id AS "Node ID",
n.store_id AS "Store ID",
round(u.content_size/1024/1024,2) AS "Size (MB)",
n.uuid AS "Document ID (UUID)",
n.audit_creator AS "Creator",
n.audit_created AS "Creation Date",
n.audit_modifier AS "Modifier",
n.audit_modified AS "Modification Date",
p1.string_value AS "Document Name",
u.content_url AS "Location"
FROM alf_node AS n,
alf_node_properties AS p,
alf_node_properties AS p1,
alf_namespace AS ns,
alf_qname AS q,
alf_content_data AS d,
alf_content_url AS u
WHERE n.id=p.node_id
AND ns.id=q.ns_id
AND p.qname_id=q.id
AND p.long_value=d.id
AND d.content_url_id=u.id
AND p1.node_id=n.id
AND p1.qname_id IN (SELECT id FROM alf_qname WHERE local_name='name')
AND u.content_url='store://2015/4/30/12/5/0e111f05-7fcf-4a44-b719-b94cd04dd5ab.bin';

II. Number of…

From a reporting point of view, let’s say that you need some information regarding the number of… something. In this case and if you want to use your DB directly, then there is a really simple solution (simple but is it the best?) because Alfresco provide a Database architecture that is quite simple to understand and to use to get what you need. Indeed, if you take a look at the “alf_qname” table, you will see that every element that is part of Alfresco has its QName listed here. A QName is the Qualified Name of a repository item. This can be seen as a kind of “Super-Type”:

alfresco=> SELECT * FROM alf_qname;
 id | version | ns_id | local_name
----+---------+-------+------------
  1 |       0 |     1 | store_root
  2 |       0 |     1 | aspect_root
  3 |       0 |     1 | container
  4 |       0 |     1 | children
  5 |       0 |     2 | user
...
 24 |       0 |     6 | folder
...
 51 |       0 |     6 | content
...
133 |       0 |     6 | thumbnail
134 |       0 |    13 | rendition
...

As you can see above, if you are searching for something that has a content, it can be done quite easily using the id or the local_name that correspond to that. So based on this table, here are some queries that can be useful:

Retrieve the number of users in the Repository

SELECT count(*)
FROM alf_node AS n,
alf_qname AS q
WHERE n.type_qname_id=q.id
AND q.local_name='user';

Retrieve the number of elements with a content in the Repository (include system’s documents)

SELECT count(*)
FROM alf_node AS n,
alf_qname AS q
WHERE n.type_qname_id=q.id
AND q.local_name='content';

Retrieve the number of thumbnails in the Repository

SELECT count(*)
FROM alf_node AS n,
alf_qname AS q
WHERE n.type_qname_id=q.id
AND q.local_name='thumbnail';

Retrieve the number of renditions in the Repository

SELECT count(*)
FROM alf_node AS n,
alf_qname AS q
WHERE n.type_qname_id=q.id
AND q.local_name='rendition';

Of course you can do that for all QNames but you can also be more precise! So based on the query to retrieve the number of elements with a content, if you only want the number of documents of a specific type, then you can simply complete your query:

Retrieve the number of XML documents in the Repository

SELECT count(*)
FROM alf_node AS n,
alf_qname AS q,
alf_node_properties AS p
WHERE n.type_qname_id=q.id
AND p.node_id=n.id
AND p.qname_id IN
(SELECT id
FROM alf_qname
WHERE local_name='name')
AND q.local_name='content'
AND p.string_value LIKE '%.xml';

Retrieve the number of PDF documents in the Repository

SELECT count(*)
FROM alf_node AS n,
alf_qname AS q,
alf_node_properties AS p
WHERE n.type_qname_id=q.id
AND p.node_id=n.id
AND p.qname_id IN
(SELECT id
FROM alf_qname
WHERE local_name='name')
AND q.local_name='content'
AND p.string_value LIKE '%.pdf';

As the creation date, creator, modification date and modifier information are also stored on the “alf_node” table, you can also very easily filter your query based on the creation/update date of an Alfresco Node. That’s pretty cool, right?!

III. Lifecycle specific

To complete the relation between this blog post and the previous one, I wanted to share some queries that can be used to identify the current state of a document. As explained in my previous post, a document that is not yet deleted will be in the store named “workspace://SpacesStore”. A document that has been deleted by a user will be in the sotre named “archive://SpacesStore” and when this document is removed from the global trashcan, the orphan_time is set to the current timestamp. With all these information and with the “alf_node” and “alf_content_url” tables we can easily build our own queries to find what is needed.

alfresco=> SELECT * FROM alf_store;
 id | version | protocol  |       identifier        | root_node_id
----+---------+-----------+-------------------------+--------------
  1 |       1 | user      | alfrescoUserStore       |            1
  2 |       1 | system    | system                  |            5
  3 |       1 | workspace | lightWeightVersionStore |            9
  4 |       1 | workspace | version2Store           |           10
  5 |       1 | archive   | SpacesStore             |           11
  6 |       1 | workspace | SpacesStore             |           12
(6 rows)

So let’s find all documents that have been created and aren’t deleted yet:

All documents created in their active life

SELECT *
FROM alf_node
WHERE store_id=6
AND type_qname_id=51;

The next step on the lifecycle is when the documents have been deleted by a user but aren’t deleted from the global trashcan (orphan_time is still NULL):

All documents created that are in the global trashcan (deleted by users)

SELECT *
FROM alf_node
WHERE store_id=5
AND type_qname_id=51;

Finally, when the documents are removed from the global trashcan, some references/fields are removed, the QName of these documents change from “content” (51) to “deleted” (140) on the “alf_node” table and the orphan_time is set to the current timestamp on the “alf_content_url” table:

All elements that have been removed from the global trashcan and that are now orphaned

SELECT *
FROM alf_content_url
WHERE orphan_time IS NOT NULL;

I hope you enjoyed this blog post because it was quite hard for me to write something about database queries without giving up my soul to the DB world! See you soon ;).

Cet article Alfresco: some useful database queries est apparu en premier sur Blog dbi services.

Alfresco support, by default, the preview in your browser of some video formats like mp4 but doesn’t support some others like mkv, avi or wmv. Even if Google Chrome can read mkv files for example, if you try to use the action “View in Browser” (this is not the same thing as the preview), Chrome will not try to play the mkv file but will download it instead. That’s why if you upload an mp4 video in Alfresco, you will certainly be able to watch this video directly in your browser. In this blog, I will try to explain how to configure an external tool to take care of the thumbnails creation and transformations for your videos. To achieve that, we will install and configure ffmpeg. In addition to that, you can also very easily configure Alfresco to embed an external video player like “FlowPlayer” that would take care of playing all video formats directly “streaming” from Alfresco, not using your browser. Basically, this is done by replacing the preview page for some Mime types but I will not describe it in details here.
For this blog post, I will use /opt/alfresco-4.2.c as the $ALF_HOME environment variable: the folder where Alfresco has been installed. The current release of ffmpeg is version 2.6.3. This is at least working for Alfresco v4.2.x and v5.x.

I. Installation of ffmpeg

The ffmpeg binaries/executables are available in the ffmpeg website. From this website, download the “Static” build for Windows, Linux or Mac depending on the Operating System on which Alfresco has been installed. For this example, I will use a linux host (RedHat 6.x, 64 bits) but I can assure you that it’s working properly on Windows too. I already installed ffmpeg with several versions of Alfresco from v4.2.x to 5.x.

These commands will download, extract, change the permissions/ownership and rename the folder to something more friendly (at a more proper location too).

II. Configuration of Alfresco

Once ffmpeg has been installed, the next step is to configure Alfresco to use it. The first thing to do is to add some parameters in the well-known alfresco-global.properties file. Don’t hesitate to customize these paremeters, remove the lines that aren’t needed, aso…

In the above configuration, the parameter “ffmpeg.exe=/opt/alfresco-4.2.c/ffmpeg/ffmpeg” is the location of the binary file named ffmpeg. If you install Alfresco and ffmpeg on a Windows host, then this parameter looks like: “ffmpeg.exe=C:/Alfresco-4.2.c/ffmpeg/bin/ffmpeg.exe”
Once this is done, you need to enable the thumbnail & transformation extensions in Alfresco and this is done using the following steps:

If you want, you can parse these two files to understand what Alfresco will do with ffmpeg but basically these files define the commands and options that will be used by Alfresco.
And that’s it, Alfresco has been configured to use ffmpeg for thumbnails creation and transformations of your video. For Alfresco to take these changes into account, simply restart your Application Server using an Alfresco service:

Or using Alfresco default scripts:

To check if ffmpeg is working properly, simply open Alfresco Share, browse your repository to find a video with an avi or wmv format and a thumbnail should now be present (if not, just refresh the page…). You can also try to upload a video and if a thumbnail is created for this video, then ffmpeg is working!

Cet article Alfresco: video thumbnails and transformations est apparu en premier sur Blog dbi services.

Different versions of Alfresco can be installed with different versions of Tomcat. Alfresco provide a default version with its installer but when installing Alfresco manually, an administrator can choose to install Alfresco using a different version of Apache Tomcat. Usually, each version of Alfresco is officially supported for only one specific version of Apache Tomcat and that’s fine for some use cases but I think that most Alfresco administrators will not follow this specific recommendation, at least for the Community Edition.

You can find the list of supported versions for the Alfresco Enterprise software on the Alfresco Web Site and we can also use this as a base for the Community Edition even if there is no official support for the CE. Now the question is then which version of the Apache Tomcat should be used for the Alfresco Community Edition? I would tend to say that it is probably best to always use the latest available patch of the Apache Tomcat version that is supported by Alfresco. Here is an example for an Alfresco Community Edition 4.2.x:

1. Download the file “Supported Platforms for Alfresco Enterprise 4.2.x.pdf”
2. Based on this file, Alfresco Enterprise 4.2.0 support Apache Tomcat 7.0.42
3. Open the Apache Tomcat WebSite
4. Select the Apache Tomcat 7.0 version (left panel) and download the latest stable minor release which is 7.0.68 today

I would tend not to upgrade Apache Tomcat to the next major release (Tomcat 8 or 9) because it can imply an upgrade of the JRE and some other changes which can bring some issues with Alfresco, aso… Apache is trying to keep all minor releases fully backwards compatible which make this minor upgrade generally possible without much trouble.
If you choose the latest available minor version during the installation that’s good but after some time, you might want/need to upgrade this version for several reasons:

• Bug fix
• Security fix
• Preventive actions
• aso…

Therefore in this blog, I will show you the minimum commands to properly upgrade Apache Tomcat. These commands might change a little bit depending on your actual Operating System and versions of Alfresco and/or Tomcat but you should catch the idea. Therefore I will base my commands on one of my test environment with the following characteristics:

• RedHat Enterprise Linux 6.1
• Alfresco Community Enterprise 4.2.c installed in /opt/alfresco-4.2.c/
• Upgrade Apache Tomcat from 7.0.30 to 7.0.68

So first of all, let’s download the software:

alfresco@vmdevalf02:$ cd /opt/alfresco-4.2.c/
alfresco@vmdevalf02:$ wget http://mirror.switch.ch/mirror/apache/dist/tomcat/tomcat-7/v7.0.68/bin/apache-tomcat-7.0.68.tar.gz
alfresco@vmdevalf02:$ mv apache-tomcat-7.0.68/ tomcat-7.0.68

Once done, the idea is simply to transfer all specific customization from your old Tomcat version to the new one. This include the specific JVM parameters setup in the setenv.sh and usually, there is nothing more to do for the tomcat/bin folder. Another point to not miss is the definition of the shared.loader in the catalina.properties:

alfresco@vmdevalf01:$ cp tomcat/bin/setenv.sh tomcat-7.0.68/bin/setenv.sh
alfresco@vmdevalf01:$ sed -i 's/shared.loader=/shared.loader=${catalina.base}\/shared\/classes,${catalina.base}\/shared\/lib\/*.jar/' tomcat-7.0.68/conf/catalina.properties

After that comes the ‘big part’ which is clearly not that big! The next step in our upgrade is to modify the server.xml of the new tomcat version to include all necessary customization. Here are the commands that I executed to enable the SSL, increase the maxHttpHeaderSize for Kerberos and also apply some security best practices like hiding the versions of your Web Application Server:

alfresco@vmdevalf01:$ sed -i 's/Connector port="8080"/Connector port="8080" URIEncoding="UTF-8"/' tomcat-7.0.68/conf/server.xml
alfresco@vmdevalf01:$ sed -i 's/    redirectPort="8443"/    redirectPort="8443" maxHttpHeaderSize="32768" server="Apache Tomcat" xpoweredby="false"/' tomcat-7.0.68/conf/server.xml
alfresco@vmdevalf01:$ sed -i 's/maxSavePostSize="-1"/maxSavePostSize="-1" server="Apache Tomcat" xpoweredby="false"/' tomcat-7.0.68/conf/server.xml

alfresco@vmdevalf01:$ sed -i '/Connector port="8009"/a \
        <Connector port="8443" protocol="org.apache.coyote.http11.Http11Protocol" SSLEnabled="true" \
            maxThreads="150" scheme="https" keystoreFile="\/opt\/alfresco-4.2.c\/alf_data\/keystore\/ssl.keystore" keystorePass="keypassword" keystoreType="JCEKS" \
            secure="true" connectionTimeout="240000" truststoreFile="\/opt\/alfresco-4.2.c\/alf_data\/keystore\/ssl.truststore" truststorePass="trustpassword" truststoreType="JCEKS" \
            clientAuth="false" sslProtocol="TLS" allowUnsafeLegacyRenegotiation="true" maxSavePostSize="-1" \/>' tomcat-7.0.68/conf/server.xml

Of course, you can also open the file directly and put the lines inside it… Just remove the different non-needed ‘\’ accross the lines. If you aren’t using the access logs (for DEV/TEST environments for example), I would also recommend you to comment the Valve logging (org.apache.catalina.valves.AccessLogValve). Once done, you can check the differences between the two versions of tomcat using something like:

alfresco@vmdevalf01:$ diff tomcat-7.0.68/conf/server.xml tomcat/conf/server.xml

Next step the tomcat users. Alfresco usually define two default users that you might have and use. These users are put in the file tomcat-user.xml:

alfresco@vmdevalf01:$ cat tomcat/conf/tomcat-users.xml | grep -i Alfresco
  <user username="CN=Alfresco Repository Client, OU=Unknown, O=Alfresco Software Ltd., L=Maidenhead, ST=UK, C=GB" roles="repoclient" password="null"/>
  <user username="CN=Alfresco Repository, OU=Unknown, O=Alfresco Software Ltd., L=Maidenhead, ST=UK, C=GB" roles="repository" password="null"/>

You can simply add these two lines in the new Tomcat version (again, don’t forget the ‘\’):

alfresco@vmdevalf01:$ sed -i '/<tomcat-users>/a \
  <user username="CN=Alfresco Repository Client, OU=Unknown, O=Alfresco Software Ltd., L=Maidenhead, ST=UK, C=GB" roles="repoclient" password="null"\/> \
  <user username="CN=Alfresco Repository, OU=Unknown, O=Alfresco Software Ltd., L=Maidenhead, ST=UK, C=GB" roles="repository" password="null"\/>' tomcat-7.0.68/conf/tomcat-users.xml

This conclude the configuration part. Now you just have to copy the data from the old version to the new one:

alfresco@vmdevalf01:$ cp -R tomcat/endorsed/ tomcat-7.0.68/
alfresco@vmdevalf01:$ cp tomcat/lib/postgresql-9.0-802.jdbc4.jar tomcat-7.0.68/lib/
alfresco@vmdevalf01:$ cp -R tomcat/scripts/ tomcat-7.0.68/
alfresco@vmdevalf01:$ cp -R tomcat/shared/ tomcat-7.0.68/
alfresco@vmdevalf01:$ rm -rf tomcat-7.0.68/webapps/docs tomcat-7.0.68/webapps/examples/ tomcat-7.0.68/webapps/host-manager tomcat-7.0.68/webapps/manager
alfresco@vmdevalf01:$ cp tomcat/webapps/alfresco.war tomcat-7.0.68/webapps/
alfresco@vmdevalf01:$ cp tomcat/webapps/share.war tomcat-7.0.68/webapps/

Now depending on your Indexing system (Lucene?Solr 1? Solr4?), this next step will change… If you are using Solr4, you probably also have a solr.war file on the webapps folder and therefore you can just put that in the new version too. If you are using Solr1, you might not have a war file in the webapps folder because at the beginning, Solr was deployed in the alf_data folder and therefore there is another way to indicate to Tomcat that the Solr is deployed here:

echo "Solr war file in webapps folder:"
alfresco@vmdevalf01:$ cp tomcat/webapps/solr.war tomcat-7.0.68/webapps/

echo "Solr war file in alf_data:"
alfresco@vmdevalf01:$ cp -R tomcat/conf/Catalina/ tomcat-7.0.68/conf/

echo "Other solution... (depends on how you installed/configured Lucene/Solr for Alfresco)"

Of course, you will also need to update any other customization that you may have done like:

• Updating the logging configuration of Tomcat (conf/logging.properties)
• Change the default configured ports (conf/server.xml)
• Change the default error pages (404, 500)
• aso…

That’s why I would strongly suggest you – before to actually switch from one version to another – to manually compare all configuration files in the “conf” folders. This can be done using a tool such as notepad++ on Windows or simply a “diff <path_to_file1> <path_to_file2>” on Linux based systems.

When you are sure that everything has been copied from your old version of Tomcat to the new one, you can actually do the change:

alfresco@vmdevalf01:$ service alfresco stop
alfresco@vmdevalf01:$ mv tomcat/ tomcat-7.0.30/
alfresco@vmdevalf01:$ mv tomcat-7.0.68/ tomcat/
alfresco@vmdevalf01:$ service alfresco start
alfresco@vmdevalf01:$ mv tomcat-7.0.30/ /home/alfresco/backup_tomcat-7.0.30

And voila, you just have to check the logs of Alfresco and Tomcat, verify that the war files have been deployed successfully and that Alfresco is Up&Running again. To be sure that everything is working properly, you should really check the basic features of Alfresco like CheckIn/CheckOut, Online Preview, Creation of new document, Search, Workflow, aso…

In addition to this upgrade, I would also suggest you – if not already done – to change the default error pages (404, 500) as said above. This can be done pretty easily. Indeed you can simply add the following inside tomcat/webapps/ROOT/WEB-INF/web.xml:

  <error-page>
    <error-code>404</error-code>
    <location>/errors/404.html</location>
  </error-page>
  <error-page>
    <error-code>500</error-code>
    <location>/errors/500.html</location>
  </error-page>

Once done, create the error folder:

alfresco@vmdevalf01:$ mkdir tomcat/webapps/ROOT/errors

And put the errors pages inside it. This is an example for the file 404.html:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">

<head>
   <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
   <style type="text/css">
   body {
      font: 13px/1.231 arial,helvetica,clean,sans-serif;
      color: #000000;
   }

   body,div,p {
      margin: 0;
      padding: 0;
   }

   div {
      text-align: center;
   }

   div.panel {
      display: inline-block;
   }
   </style>
   <title>Alfresco Share » Page Not Found</title>
</head>

<body>
   <div>
      <br><img src="/share/themes/default/images/app-logo.png"><br><br>
      <p style="font-size:150%">Page Not Found.</p>
      <br><p>The page you were trying to access hasn't been found on the server. Please try again later or contact your administrator.</p><br>
      <a href="/share">Return to Alfresco</a><br><br><br>
      <a href="http://www.alfresco.com">Alfresco Software</a> Inc. © 2005-2012 All rights reserved.
   </div>
</body>
</html>

This conclude this blog about the upgrade of the Apache Tomcat used by Alfresco. As you saw, that’s not complicated at all, it will just take some time to be done properly because we don’t want to left something behind!

Cet article Alfresco: upgrade your version of Apache Tomcat est apparu en premier sur Blog dbi services.

As an Alfresco developer/administrator, you probably already made a few mistakes in your DEV environment while trying to add new features, a custom model or something like that. The result of that mistake might be a corrupted Database or Content Store. Yes I know it’s hard to recognize it! But then what can be done to solve the issue and restore an usable installation? Well if you don’t have a backup or don’t want to do that because it will take some time to restore a clean backup and you want to work immediately then a good solution might need to actually cleanup your entire installation.

Using a few commands or a simple script, your Alfresco can be cleaned very easily and quickly to let you continue your development using a fresh and working installation. In this blog, I will therefore show you a few commands that help me to clean an Alfresco environment (Community or Enterprise).

Basically when talking about corruption in Alfresco, here are the different components that can be affected:

• Database
• Content Store (documents in File System)
• Search Index
• War Files & Extensions (AMPs & extension + web-extension folders) – Not covered in this blog as this kind of corruption will be solved by deploying a corrective version of the AMP or removing the faulty configuration in the extension or web-extension folders

Cleaning an Alfresco installation is in fact just like simulating a corruption of all components mentioned above without any backup available and therefore some necessary steps need to be taken to solve this corruption.

So let’s start our simulation with the first bullet point. The first thing to do is to stop the Alfresco installation and only let the Database running. Depending on how you built your Alfresco Installation, the commands can differ a little bit: are you using one service for Alfresco + Database or one service for Alfresco or no service at all? On my DEV environment, one service is managing all Alfresco components:

alfresco@vmdevalf01:$ service alfresco stop
alfresco@vmdevalf01:$ service alfresco start postgresql

As you can see above, I’m using postgresql. Of course if you are using MySQL or any other Database, please adapt the commands accordingly! When Alfresco is stopped, the next step is to drop and recreate the database. This can be done using scripts provided by Alfresco:

alfresco@vmdevalf01:$ /opt/alfresco/postgresql/bin/dropdb alfresco
       => Password asked

alfresco@vmdevalf01:$ /opt/alfresco/postgresql/bin/createdb alfresco
       => Password asked

Or manually:

alfresco@vmdevalf01:$ /opt/alfresco/postgresql/bin/psql
postgres=# DROP DATABASE alfresco;
postgres=# CREATE DATABASE alfresco WITH OWNER alfresco;
postgres=# GRANT ALL PRIVILEGES ON DATABASE alfresco TO alfresco;
postgres=# \q

If you are using MySQL instead of PostgreSQL, then it will be something like that:

mysql> drop database alfresco;
mysql> create database alfresco default character set utf8 collate utf8_bin;
mysql> grant all on alfresco.* to 'alfresco'@'localhost' identified by 'alfresco_pwd' with grant option;

Now the Database has been cleaned and you need to align the File System to avoid inconsistencies (bullet points 2 and 3). This is really simple since we just remove the files. The following commands assume that the war file of solr4 has been deployed in the webapps folder of Tomcat and this folder /opt/alfresco/alf_data/solr4 only contain the model and index:

alfresco@vmdevalf01:$ rm -rf /opt/alfresco/alf_data/contentstore/*
alfresco@vmdevalf01:$ rm -rf /opt/alfresco/alf_data/contentstore.deleted/*
alfresco@vmdevalf01:$ mv /opt/alfresco/alf_data/solr4 /opt/alfresco/alf_data/backup_solr4_$(date '+%Y%m%d_%H%M')

And… That’s all! Now you just have to restart your service and Alfresco will populate the database and create the necessary elements for the indexing during the first startup:

alfresco@vmdevalf01:$ service alfresco stop postgresql
alfresco@vmdevalf01:$ service alfresco start

It may take one or two minutes to boot the first time but this is usually much more faster than restoring a complete database schema! By doing that, you will of course loose any document stored in Alfresco but for a development phase, that’s usually OK.

Note: Just to let you know, doing that for an Alfresco Enterprise Edition (or Alfresco One) with a Trial License will also reset the counter to: 30 days remaining. That’s why I would recommend you to only do that with the Alfresco Community Edition on your DEV environment or if you have a valid License Key because I don’t really know if that’s authorized by Alfresco!

Cet article Alfresco: cleanup your dev environment est apparu en premier sur Blog dbi services.

BeeCon? Beacon? Bacon? Is it some kind of party where you eat some bacon the whole day? Well… Sort of but it is much better! So let me present you the reason behind the BeeCon first.

Back in the Alfresco Summit 2013, the idea of an organization, a group or people from the Alfresco Community came up. This idea grew up and finally in 2014, the Order Of The Bee (OOTB) was created. You can find more details about this community on the website directly. In case you never noticed, the logo of Alfresco really looks like a flower and therefore we are all bees around the flower ;). I joined this community one year ago because I’m working with the Alfresco products (including the Alfresco Community Edition) for years now and I’m still loving it!

Last year, Alfresco cancelled its Alfresco Summit (3 days annual event) because they rather wanted to focus on small events all over the worlds, the Alfresco Days (0.5 or 1 day event in a lot of cities over the year). Since these Alfresco Days are, in my opinion, more sales oriented, the Order Of The Bee started to talk a few months ago about the possibility to create its own Community event, completely oriented on technical topics. And finally here we are, the first BeeCon is taking place in Brussels, the city of beer. You can find the agenda and some other information on the dedicated website too.

For the first day of this first edition, I attended some really nice sessions. The day began with a short introduction by key members of the OOTB and then a one hour keynote from John Newton, the CTO and Founder of Alfresco (as well as co-founder of Documentum) about the journey to a digital world during which he made a few announcement regarding Solr 6, AngularJS, aso… AngularJS is a JavaScript framework and apparently Alfresco is working with that to give the ability to Afresco Administrators and Developers to create their own Application (UI for Alfresco) just like “Share” (the default one). They only started to work on this topic a few weeks ago but we should expect a first prototype to be released to the community in the next month or so. I can’t wait to be able to play with this new tool to create my own Application on top of the Alfresco Repository! Some information:

After that, I attendee some presentations about how to better use PostgreSQL for Alfresco and another one regarding the AAAR module which is basically a simple way to use Business Intelligence in Alfresco using data stored in Alfresco (metadata from your database) but also data from other locations.

The Lightning Talk #1 (7 presentations of 5 minutes with a common slideshow for all presenters and with a timer that never stops) was also really interesting because we got several presentations showing that the community is very busy: new add-ons are constantly in development to create some auditing reports on Aikau or to enable the automatic recognition and classification of your documents thanks to a Machine Learning engine.

Finally, this day ended with a Q&A session with the head of Alfresco and a presentation of Thomas DeMeo (VP of Product Management @ Alfresco) regarding the RoadMap of Alfresco products for the next months. Here are two photos taken during the day to show you what is coming in the next few months/years regarding Alfresco (1st: John Newton // 2nd Thomas DeMeo):

See you tomorrow!

Cet article BeeCon – Day 1 – The place to be for Alfresco est apparu en premier sur Blog dbi services.

I didn’t say that yesterday but the BeeCon is a two days event. Yesterday was the first day and therefore the BeeCon is now officially over… I’m quite sad because these two days were really great and I would have wanted them to continue a little bit more.

I was finally able to meet some people from the Order Of The Bee with whom I had quite interesting discussions over the past few months and to see again some others which usually went to the Alfresco Summit, just like me. So in conclusion, if you are interested in Alfresco, I can only encourage you to join the OOTB and participate as much as possible. Trust me, I know it can be difficult because time is a wanted resource and contrary to the Dalton Brothers (Lucky Luke, Belgian, yes I know that was easy), you can’t really catch it whenever you want but just do what you can and it will be fine ;).

If you are reading this, then you probably read the title of this blog (if not, shame on you ! ). So let’s talk about the first session of the day presented by Joram Barrez (co-founder of the Activiti Project and Engineer @ Alfresco). As you probably already understood, his session was about the next evolution of BPM Awesomeness: Activity 6. Activity is the BPM engine included in Alfresco and starting Q4 2014/Q1 2015, Alfresco also deliver it as a separate server for you and your business architects to be able to design your own workflow from A to Z using the graphical interface only. As shown yesterday (see photos from this blog), Activiti 6 isn’t yet released since it is planned for Q4 2016. Therefore that was a pretty good taste of what will come soon with basically the presentation of what has been changed on the core product. One important point repeated during this session was that Activiti 6 will be fully backward compatible which is really cool. There are also some performance testing going on but so far Activiti 6 is always faster that the version 5.

In addition to that, there were two sessions about the Alfresco Software Development Kit (SDK) and the Alfresco Software Provisioning Kit (SPK). The first one, presented by Ole Hejlskov which shown the good, the bad and the ugly about the SDK because yes that’s true that the Alfresco SDK made some really huge improvements over the past few years but the destination is still far away… Ole also presented the past, the present and the future of the Alfresco SDK, introducing the SDK 3.0 which should come in the next few months (first release you will be able to play with will come around June). The second presentation has been given by Enzo Rivello. The Alfresco SPK can be used to automate the creation of images and systems ready to be deployed/used. It appears to be a really promising tool so take a look at that in the near future too!

I also attendee two sessions talking about migration tools. The first session was related to general considerations when using an ETL approach to perform your migration. This is actually what we did years ago when we migrated our old & crappy DMS to Alfresco (of course we are using Alfresco, that’s the best ). The second one focused on Move2Alf which is an open-source software solution to migrate to Alfresco that allows error handling, reporting, scheduling and a few other stuff. That was also quite interesting because even if you have your own way to migrate your content, comparing solutions can only bring new ideas and the solution to use will depends on the use case.

Now last but not least, zombies… Yeah, I know… I was also questioning myself when I saw in the agenda a session presented by Jeff Potts with the following title: “Would the commercial open source software you depend on survive a zombie apocalypse”. If you are a little bit interested in Alfresco, then you probably know Jeff. His session was actually quite fun because the whole reason for this presentation was: what would happen if all employees of Alfresco were attacked by zombies. The idea is simple but Jeff really made us think about it and takes us to his imaginary world. How would the community react in such case? Would we be able to take the code of Alfresco and make it lives? What are the strength of the community and Alfresco and what can be done to improve that in the future?

I also attendee some others sessions like the following ones but I can’t talk about all sessions even if I would like to:
– Lightning Talk #2
– What to think about when you’re planning to install Alfresco
– The product vision for Community Edition
– …

Still not convinced by the BeeCon? Well take a look at the few goodies everybody got and I’m sure you will be there too for the next one because yes there will be a next one!

Especially the glass and the M&M’s marked “BeeCon 2016″, that’s soooo cool :D.

I wasn’t able to attend the last two sessions because the Brussels’ airport is quite a mess at the moment… Instead of ~ one hour to go through the security, it actually took me a little bit less than three hours so plan accordingly… Even then, my trip to Brussels was a success and I’m looking forward to do it all over again next year (in Spain :D)!

See you around ;).

Cet article BeeCon – Day 2 – Activiti, SDK/SPK, Migration and Zombies est apparu en premier sur Blog dbi services.

In this blog, I will talk about the setup of an Apache httpd server with/without SSL configuration on top of your Alfresco installation. This way of configuring Apache httpd in front of your Web Application can be reused very easily for all other applications like your intranet, competency center, resource planning, aso… It is NOT specific to Alfresco and this is what makes this blog very useful!

When you install Alfresco using the installer provided, you will get a tomcat setup on port 8080 (by default) which is also listening on port 8443 with an SSL Certificate generated by Alfresco for the communications between the Alfresco repository and the Solr engine. You can use this SSL Certificate to secure your Alfresco installation but there are two restrictions:

• The password used for this SSL Certificate is well known since it is always the same… You just have to download the installer, run it and you will be able to find the password => Not really secure!
• The SSL Certificate is self-signed by Alfresco, which means that it will not be trusted in your company => SSL Warnings/Errors

Based on these information, you have several choices:

• Generate your own Self-Signed SSL Certificate with a new password that nobody knows and replace the SSL Certificate used by Alfresco by default => Secure but still a Self-Signed SSL Certificate! In addition, there are some documentation about how to do that but that’s not so easy and trivial…
• Generate your own SSL Certificate using a trusted Certificate Authority and replace the SSL Certificate used by Alfresco by default => Secure but the setup is still not so easy and trivial…
• Generate your own Self-Signed SSL Certificate and setup something in front of Alfresco to handle the SSL communications => Secure, easier but still a Self-Signed SSL Certificate!
• Generate your own SSL Certificate using a trusted Certificate Authority and setup something in front of Alfresco to handle the SSL communications => Secure, easier, trusted!

So yes, you already understood it I guess, I will show you in this blog how to setup the last option with/without the SSL part. With this choice, you will have something really easy to setup because you don’t need to change anything on Alfresco side! You can let your Tomcat running on port 8080, 8443 and you just don’t care about the internal communications between the Alfresco repository and the Solr engine.

For this blog, I used a RedHat Enterprise Linux 6.4. If you are using a different version or a different distribution, just adapt the few steps that will not work for you (basically the “su -” or “yum” commands). In addition to that, I will define the following:

• OS: RHEL 6.4
• Hostname: vmdevalfresco01
• Domain: your-domain.com
• SSL Certificate: *.your-domain.com    (wildcard certificate => You can use something else of course!)
• Alfresco running on Apache Tomcat 7.0.68 (the version doesn’t really matter!)

So let’s start this setup by switching to root for the entire installation:

[alfresco@vmdevalfresco01 ~]$ su -
Password:

Now we can install the Apache httpd server and setup some prerequisites:

[root@vmdevalfresco01 ~]$ yum install httpd*
[root@vmdevalfresco01 ~]$ yum install gcc httpd-devel mod_ssl

Once Apache httpd server is installed, you have to verify that the “mod_so.c” is present in your machine. To do that, just execute the following command and ensute that it is present in the list:

[root@vmdevalfresco01 ~]$ apachectl -l
Compiled in modules:
  core.c
  prefork.c
  http_core.c
  mod_so.c

Ok so now Apache httpd is installed (not configured) so we can download the Tomcat Connector. This little module will basically take care of all communications between the Apache httpd server and your Apache Tomcat on which Alfresco is running:

[root@vmdevalfresco01 ~]$ cd /tmp
[root@vmdevalfresco01 tmp]$ wget http://archive.apache.org/dist/tomcat/tomcat-connectors/jk/tomcat-connectors-1.2.40-src.tar.gz
[root@vmdevalfresco01 tmp]$ tar -xvf tomcat-connectors-1.2.40-src.tar.gz
[root@vmdevalfresco01 tmp]$ cd tomcat-connectors-1.2.40-src/native

At the moment, the source of the Tomcat Connector is on your file system but you still need to build the Apache httpd module! But before that, you will need to check where your apxs is installed (apxs = APache eXtenSion tool). You can use the which command or find it yourself:

[root@vmdevalfresco01 native]$ which apxs
/usr/sbin/apxs
[root@vmdevalfresco01 native]$ find / -name apxs
/usr/sbin/apxs

Normally apxs is installed when you install the Apache httpd server but if it is not the case, then you need to install it manually. Once everything is ready, you can actually build the Tomcat Connector (of course change the path to the apxs if it differs):

[root@vmdevalfresco01 native]$ ./configure --with-apxs=/usr/sbin/apxs
[root@vmdevalfresco01 native]$ make

The next step is to check where your Apache httpd modules are installed (by default it is under /usr/lib64/httpd/modules but you can also use the Symlink /etc/httpd/modules directly). Then copy the new Tomcat Connector module (mod_jk.so) in this Apache httpd module folder:

[root@vmdevalfresco01 native]$ ls -l /etc/httpd/
[root@vmdevalfresco01 native]$ cp apache-2.0/mod_jk.so /usr/lib64/httpd/modules/

Ok so the Tomcat Connector module is now in the right location but there are two remaining things to do. The first one is to configure a worker and this is done by creating a file workers.properties with the following content:

[root@vmdevalfresco01 native]$ cd /etc/httpd/
[root@vmdevalfresco01 httpd]$ cat /etc/httpd/conf.d/workers.properties
worker.list=worker1
worker.worker1.type=ajp13
worker.worker1.port=8009
worker.worker1.host=localhost
worker.worker1.lbfactor=1

Of course for the worker to work properly, the AJP13 Connector in Tomcat must be enabled and configured to use the port 8009 (default one). The second and last thing to do for the Tomcat Connector is to actually load the module that we built into the Apache httpd server and let Apache know where the worker is defined:

[root@vmdevalfresco01 httpd]$ cat /etc/httpd/conf.d/mod_jk.conf
LoadModule      jk_module modules/mod_jk.so

JkWorkersFile   conf.d/workers.properties
JkLogFile       logs/mod_jk_log
JkLogLevel      info
JkMount         /* worker1

At this point, we are almost done! The next step is to actually configure your Apache httpd server as you want/need. On the file /etc/httpd/conf/httpd.conf, you can change what you want but you will probably want to update the following values:

• ServerTokens Prod
• KeepAlive On
• Listen 80
• Include conf.d/*.conf
• User apache
• Group apache
• ServerAdmin firstname.lastname@your-domain.com
• ServerName vmdevalfresco01.your-domain.com:80
• ServerSignature Off
• aso…

I. First solution – Apache httpd server in HTTP

Now comes the time to configure your Apache httpd server to work in combination of Alfresco. If you want to use Apache in HTTP (I really recommend you to set it up in HTTPS…), you just have to add the following lines at the end of the file /etc/httpd/conf/httpd.conf (or create a conf file in the conf.d folder):

[root@vmdevalfresco01 httpd]$ tail -10 /etc/httpd/conf/httpd.conf
<VirtualHost *:80>
    ServerName vmdevalfresco01.your-domain.com
    ServerAlias vmdevalfresco01 vmdevalfresco01.your-domain.com
    RewriteLog /var/log/httpd/vmdevalfresco01.log
    RewriteLogLevel 5
    RewriteEngine On
    RewriteRule ^/$ http://vmdevalfresco01.your-domain.com/share [R,L]
    JkMount /* worker1
</VirtualHost>

These few lines will basically redirect all requests for “http://vmdevalfresco01.your-domain.com” to “http://vmdevalfresco01.your-domain.com/share”. It is a simple redirection but then Alfresco will be able to catch it because by default Alfresco isn’t using the context “/” but is using “/share” (or “/alfresco”). In addition to that, there is also the definition of a Worker for our mod_jk and this is what will actually do the communications between the Apache httpd server and your Apache Tomcat. With this configuration, you can now start your Apache httpd server:

[root@vmdevalfresco01 httpd]$ service httpd start

You can now access “http://vmdevalfresco01.your-domain.com” and it will in the background use your Apache Tomcat defined in the port 8080 (or any other ports) without showing it to the end users.

II. Second solution – Apache httpd server in HTTPS

If you want to use Apache httpd server in HTTPS which is the recommended way to go, it will requires a little bit more configuration. The first thing to do is to redirect all HTTP communications to HTTPS. For that purpose, add the following lines at the end of the file /etc/httpd/conf/httpd.conf (or create a conf file in the conf.d folder) as done for the HTTP part but the RewriteRules are different:

[root@vmdevalfresco01 httpd]$ tail -11 /etc/httpd/conf/httpd.conf
<VirtualHost *:80>
    ServerName vmdevalfresco01.your-domain.com
    ServerAlias vmdevalfresco01 vmdevalfresco01.your-domain.com
    RewriteLog /var/log/httpd/vmdevalfresco01.log
    RewriteLogLevel 5
    RewriteEngine On
    RewriteRule ^/$ https://vmdevalfresco01.your-domain.com/share [R,L]
    RewriteRule ^/?(.*) https://vmdevalfresco01.your-domain.com/$1 [R,L]
    JkMount /* worker1
</VirtualHost>

As you can see above, the main difference with the configuration done for the HTTP setup is that there are two rewrite rules now and both are redirecting to the HTTPS URL. In addition to that, you will also have to edit the file /etc/httpd/conf.d/ssl.conf which has been created with a default content when installing the module mod_ssl. You can setup the following content:

[root@vmdevalfresco01 httpd]$ cat /etc/httpd/conf.d/ssl.conf
LoadModule ssl_module modules/mod_ssl.so

Listen 443

SSLPassPhraseDialog  builtin

SSLSessionCache         shmcb:/var/cache/mod_ssl/scache(512000)
SSLSessionCacheTimeout  300

SSLMutex default

SSLRandomSeed startup file:/dev/urandom  256
SSLRandomSeed connect builtin

SSLCryptoDevice builtin

<VirtualHost *:443>
    ServerName vmdevalfresco01.your-domain.com
    ServerAlias vmdevalfresco01 vmdevalfresco01.your-domain.com
    ErrorLog logs/ssl_error_log
    TransferLog logs/ssl_access_log
    LogLevel warn

    RewriteLog /var/log/httpd/vmdevalfresco01-ssl.log
    RewriteLogLevel 5
    RewriteEngine On
    RewriteRule ^/$ https://vmdevalfresco01.your-domain.com/share [R,L]
    JkMount /* worker1

    SSLEngine on
    SSLProtocol All -SSLv2 -SSLv3
    SSLCipherSuite EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:AES256+RSA
    SSLHonorCipherOrder on
    SSLCertificateFile /etc/pki/tls/certs/wildcard_your-domain_com.crt
    SSLCertificateKeyFile /etc/pki/tls/private/wildcard_your-domain_com.key
    SSLCertificateChainFile /etc/pki/tls/certs/gd_bundle-g2-g1.crt

    <Files ~ "\.(cgi|shtml|phtml|php3?)$">
        SSLOptions +StdEnvVars
    </Files>
    <Directory "/var/www/cgi-bin">
        SSLOptions +StdEnvVars
    </Directory>
    SetEnvIf User-Agent ".*MSIE.*" \
    nokeepalive ssl-unclean-shutdown \
    downgrade-1.0 force-response-1.0
    CustomLog logs/ssl_request_log \
    "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
</VirtualHost>

Some notes about the configuration setup in the file ssl.conf:

• SSLProtocol = All weak SSL Protocols (SSLv2 and SSLv3) have been deactivated.
• SSLCipherSuite = Only a specific list of Ciphers has been setup and therefore only communications with these Ciphers will be enabled. That way, we are sure that a hacker will not be able to use weak encryption mechanisms.
• SSLCertificateFile = This is the SSL Certificate that Apache httpd will use.
• SSLCertificateKeyFile = This is the Private Key used to generate the SSL Certificate.
• SSLCertificateChainFile = This is the SSL Certificate Trusted Chain. In case you have a Root CA and a Gold CA, then this file should contain the complete chain that will ensure the SSL Certificate to be trusted. This is usually provided by your Certificate Authority.

With this configuration, all HTTP and HTTPS communications will be redirected to HTTPS only (“http://vmdevalfresco01.your-domain.com/share”). In addition to that, everything said for the HTTP also apply for the HTTPS setup. Once this is done and everything is in place, you can now start your Apache httpd server:

[root@vmdevalfresco01 httpd]# service httpd start

You can now access “http://vmdevalfresco01.your-domain.com” or “https://vmdevalfresco01.your-domain.com” or “https://vmdevalfresco01.your-domain.com/share” and the Apache httpd server will redirect everything to HTTPS and use in the background use your Apache Tomcat defined in the port 8080 (or any other ports) without showing it to the end users.

And that’s it!

Cet article Alfresco – Protect your installation with Apache httpd est apparu en premier sur Blog dbi services.

Introduction

Nowadays companies have to deal with lots of electronic documents, some of them being mission critical. Insurances, Banks and Pharma industries are good candidates for ECM/DMS solutions since they produce and deal with lots of documentations, contracts and receipts. Usually the largest ECM/DMS infrastructures can be found at those customers which initiate large digitalization processes. However even for smaller businesses, managing e-documents like sales quotations, offers, answers to RFIs and RFPs becomes mission critical. Indeed, while creating such quotations and offers, collaboration is often requested between salesmen and eventually with the technical department too. The ECM/DMS solutions must offer the means to share and work together on the same document. Unfortunately these documents are, most of the time, simply lying around on a basic Windows Share, if the users even took the time to copy the documents on this share. As a result, there is no concurrency management, preventing any data loss and the “locking strategy” is quite simple: “last wrote … won”. It’s even incredible to see how many “larger” companies still work like that. All companies follow the digitalization trends but sometimes in a quite elementary way.

So basically what prevents the usage of a ECM/DMS solution in all companies? From my point of view, most of the time ECM/DMS projects are wrongly sized and approached. Indeed, each customer has lots of good intentions at the begin of the project. Therefore instead of focusing on the essentials, project responsible want to implement almost everything, and may be too much:

• workflow management
• complex user/group management and security rules
• full text indexing
• infrastructure redundancy
• full integration in existing solutions
• complex business processes (mixing up BPM and ECM/DMS)
• aso … aso …

As a result the proposed ECM/DMS solutions can become quite complex to set up and quite expensive in terms of licenses. That’s exactly where those kinds of projects usually get stuck and die. We want to do too much, it gets too complex, so let’s do nothing!

Is there a way and a technology which allows to start smoothly in the ECM/DMS area?

Why a DMS?

First of all, let’s summarize again which core functionalities we need from a DMS. In other words, what do we want to achieve with such a solution?

As a salesman, and in particular as a Chief Sales Officer, I need to keep a clear track of all changes. Indeed, while editing/changing documents, and in particular quotations, we should keep traces of each modifications. Release management and traceability is a “must have” nowadays. Document validation (workflow processes) would be nice to have in a second step.

Of course in the current context of cyber-attacks, I need a high security level. I also need to protect the documents against unauthorized users: we do not want/need all people in the company to know the sales engineering policy. Furthermore, we do not want viruses encrypting all our documents lying on a simple Windows Share. If the ECM/DMS solutions request identifications to the system to proceed with CheckOut/CheckIn procedures to work on documents the virus has almost no chance to access easily all files.

If this CheckOut/CheckIn procedure is included in the Microsoft Office suite, it won’t even decrease the efficiency of the users or of the salesmen. Users are always afraid when they have to do more that simple double clicks

Why Alfresco?

As explained in the introduction, the costs and the over sizing of ECM/DMS projects may sometimes kill them before they even born.

Alfresco is an Open Source ECM/DMS solution allowing to implement quite quickly and easily the core needed functions without license costs. Of course, the Enterprise version offers some additional features like:

• content encryption at rest and encrypted configuration files
• clustering
• synchronization of content between Cloud and On-Premises installations

At dbi services, since we are ECM/DMS experts we decided to implement Alfresco on our own. However, the setup and documentation of such a solution can be limited to several days, not weeks or years. We do not need bunch of senior technical experts and presales over several months to set it up, like for some un-named ERP solutions

Out of the box, and in particular with the version 5.x, Alfresco really covers 100% of what I do expect from an ECM/DMS, as a salesman:

• integrated release management
• protection and management of concurrency between users
• protection against viruses since some identification is needed and you can always revert a previous version if needed
• easy drag & drop functionality to copy documents/folders into alfresco

Below, an example of the smooth integration of Alfresco in each Small and Medium Businesses environment using MS Office. With a smooth integration in MS Office it is now possible to directly work on a document and save it into alfresco without having to “CheckOut/CheckIn” it, since this operation is integrated in the Office connector. Below an example of the integration of Alfresco in MS Office once a so called “SharePoint online location” (compatible with alfresco) has been created. you can directly open the documents in Word from the Alfresco repository (checkin/checkout happens in the background) :

Another example of smooth integration in the MS or Mac world, the drag and drop feature from the file browser directly in the Alfresco browser using any Web browser :

It is even possible to save a newly created MS Office document directly into Alfresco, the integration has been really maximized in the last Alfresco release (5.x).

Another strong advantage of Alfresco is basically coming from the Open Source culture. Despite the fact that some companies still have the feeling that they have to pay expensive software licenses, it may sometimes be possible to think about the “service only” model. This approach, used by Open Source software, allows the product to improve and growth through contributors offering their services around the product. That’s the case for dbi services providing support around Alfresco which allows a professional usage of the solution. In the same idea, lots of contributors developed some Alfresco extensions allowing to improve the core functionalities and to integrate the product in lots of other solutions or products (i.e. in ERP solutions like Odoo, SAP, Salesforce, aso…). Some of these add-ons that were developed by the community are even integrated directly into the next Alfresco releases to improve the product (HTML5 Previewer, Trashcan Cleaner, aso…).

Providing the complete set of required core features, easy to deploy, manage and administrate, cost efficient and extensible, Alfresco has become a kind of optimum choice for our company development while insuring the quality of our Sales activities.

Conclusion

As for each IT project, we do strongly advice to follow a pragmatic way, ideally proceeding with POCs (Proof Of Concepts), in order to validate, step by step, the solution. Furthermore, it is advised to focus on essential functionalities first, avoiding huge and complex specifications giving the impression that we will never reach the end of the project.

Combining efficiency and security and providing the required features, Alfresco was the most competitive price/feature solution which helped us to growth as we did over the last years. The last version we just migrated to (version 5.2) did even increase the user acceptance since the integration into the usual office tools has been seriously improved.

Cet article Alfresco 5.2 our DMS of choice! est apparu en premier sur Blog dbi services.

As you might now if you are following our blogs (see this one for example), we are using Alfresco Community Edition internally for some years now and we also have a few customers using it. Today, I will present you a small issue I faced with Alfresco which wasn’t able to display the content of a few – very specific – folders… As you will see below, it was actually not related to Alfresco but that’s a good example.

For some background on this issue, an end-user contacted me, saying that he wasn’t able to see the content of three folders in which he was sure there were at least some PDFs. So I checked and yes, even as an Alfresco Admin, I wasn’t able to see the content of this folder using Share. The only common point I could see with these three folders is that they all contained a “+” in their name. I recently upgraded and migrated this Alfresco environment so I was kind of sure this was somehow linked (since the folders existed and were working before the upgrade+migration). For this blog, I will use a test folder named “Test+Folder” and I’m using an Apache HTTPD as a front end. I put this folder under the Shared folder:

URL: https://alfresco_server_01/share/page/context/shared/sharedfiles

Clicking on this folder’s name from the central screen or from the browsertree on the left side will result in the following screen:

URL: https://alfresco_server_01/share/page/context/shared/sharedfiles#filter=path|%2FTest%2BFolder|&page=1

As you can see above, the URL accessed is the correct one, it contains the “/Test+Folder” path so that’s correct (%2F is “/” and %2B is “+”). However, the screen is empty, just like if this path would not exist and on the breadcrumb, it is shown “Shared Files” while it should be “Shared Files > Test+Folder”. So in summary, the Share UI isn’t able to display the content of this folder. For testing purpose, I accessed this folder using WebDAV and AOS and these clients were able to see the content of the folder but not Share. Like I mentioned above, this Alfresco environment is using Apache HTTPD as a front-end and then mod_jk for the communication with the Tomcat. You can take a look at this blog or the official documentation for more information on this setup.

Since other clients were working properly, I tried to access Alfresco Share without going through the front-end (so accessing Tomcat directly) in order to ensure that the issue isn’t with Alfresco itself. By doing so, I was able to see the PDFs. If the issue isn’t with Alfresco, then it should be with the front-end and in particular with the mod_rewrite and mod_jk, in this case. The idea here is to check what are doing these two mods and then compare the outcome with the initial request and what is being transferred to the Tomcat using the access logs.

While doing this analysis, I found out that the mod_jk was most probably the reason of this issue. When mod_jk is doing its job, it will decode the URL’s encoded characters like “/” instead of %2F, like “+” instead of %2B, like a space instead of %20, aso… Then once the rules have been processed, it will, by default, re-encode these special characters before transmitting the request to Tomcat. However in the Tomcat access logs, it appeared that the other special characters were indeed present in their encoded format but it wasn’t the case for the “+” sign which was shown like that (so no %2B anymore).

This is an example (it’s not the real requests in the background but this is the URL on a Web browser so it gives a good example):

• URL accessed:                    https://alfresco_server_01/share/page/context/shared/sharedfiles#filter=path|%2FTest%2BFolder|&page=1
• URL decoded by mod_jk:  https://alfresco_server_01/share/page/context/shared/sharedfiles#filter=path|/Test+Folder|&page=1
• URL sent to Tomcat:          https://alfresco_server_01/share/page/context/shared/sharedfiles#filter=path|%2FTest+Folder|&page=1

The mod_jk does not re-encode the “+” sign and this prevents the Tomcat in the back-end to complete the request properly.

Edit – 23-Aug-2017: As mentioned by alxgomz in the comments (thx for the additional information!), this is actually an understandable behavior because of this.

The behavior of mod_jk related to the URIs is managed using the JkOptions. This property can have the following values regarding the Forwarded URI:

• JkOptions     +ForwardURIProxy
  • Default value in version > mod_jk 1.2.23
  • The forwarded URI will be partially re-encoded after processing inside Apache and before forwarding to Tomcat
• JkOptions     +ForwardURICompatUnparsed
  • Default value in version = mod_jk 1.2.23
  • The forwarded URI will be unparsed so no decoding nor re-encoding
• JkOptions     +ForwardURICompat
  • Default value in version < mod_jk 1.2.23
  • The forwarded URI will be decoded by Apache
• JkOptions     +ForwardURIEscaped
  • The forwarded URI will be the encoded form of the URI used by ForwardURICompat

I’m using a fairly recent version of mod_jk on this Alfresco environment so the default value is “ForwardURIProxy”. Therefore on the paper, it should work properly since URIs will be decoded and re-encoded… However we saw above that this is working but not for the “+” sign which is not re-encoded.

To workaround this issue, I just updated the JkOptions to have “JkOptions     +ForwardURICompatUnparsed” in my Apache HTTPD configuration and after a reload of the conf, I was able to access the content of the folder:

URL: https://alfresco_server_01/share/page/context/shared/sharedfiles#filter=path|%2FTest%2BFolder|&page=1

Please note that “ForwardURICompatUnparsed” will always forward the original request URI, so rewriting URIs with mod_rewrite might not work properly, it all depends how you configured the rewrite rules and what you need to do with it. So basically if you don’t have this issue, I would suggest you to use the default JkOptions.

Cet article Alfresco – Unable to see the content of folders with ‘+’ in the name est apparu en premier sur Blog dbi services.

When playing with the AOS implementation, a colleague of mine faced an interesting issue where he just wasn’t able to move or rename any files or folders. The creation and deletion were working properly but he was unable to move or rename anything when using a Network Location or a Network Drive. This environment was freshly installed with a front-end (Apache HTTPD) setup in SSL so we worked together to find out what was the issue. All workstations were impacted no matter what OS was used (Windows 7, 8, 10, aso…).

The Network Location or Drive were mounted using the following:

• URL Style => https://alfresco_server_01.domain.com/alfresco/aos/
• WebDAV Style => \\alfresco_server_01.domain.com@SSL\DavWWWRoot\alfresco\aos\

In all cases, the workstations were able to connect and create nodes in the Alfresco Server (through AOS), it meant that the parameters/configuration on the Alfresco and Apache HTTPD sides were pretty much OK but nevertheless we still checked them to be sure:

alfresco@alfresco_server_01:~$ grep -E "^alfresco\.|^share\." $CATALINA_HOME/shared/classes/alfresco-global.properties
alfresco.context=alfresco
alfresco.host=alfresco_server_01.domain.com
alfresco.port=443
alfresco.protocol=https
share.context=share
share.host=alfresco_server_01.domain.com
share.port=443
share.protocol=https
alfresco@alfresco_server_01:~$
alfresco@alfresco_server_01:~$
alfresco@alfresco_server_01:~$ grep aos $CATALINA_HOME/shared/classes/alfresco-global.properties
aos.baseUrlOverwrite=https://alfresco_server_01.domain.com:443/alfresco/aos
#aos.sitePathOverwrite=/alfresco/aos
alfresco@alfresco_server_01:~$

For me, the parameters above seemed correct at first sight. The Apache HTTPD being in SSL on the port 443 and redirecting everything to the Tomcat using the mod_jk, it is normal for the alfresco and share parameters to use https and the port 443 (and even if the Tomcat is actually not in SSL) because these values should reflect the front-end. For the “aos.baseUrlOverwrite”, it is normally to be used only in case you have a proxy server in front of your Alfresco and that this proxy isn’t an Apache HTTPD. Since my colleague was using Apache, this parameter wasn’t really needed but having it set to the correct value shouldn’t hurt either. The correct value for this parameter is also the front-end URL and it is the current value, or so it seemed.

With the above parameters, we were able to create any kind of files and folders in our Network Locations/Drives. I took some screenshots for this blog and I used a simple folder to demonstrate the issue and the solution. So creating a folder with the default name is working properly:

At this point, I had a new folder in my Alfresco Server which I could clearly see and manage via the Share client. So renaming it from Share wasn’t a problem but doing the same thing through AOS (Network Location or Drive) resulted in this:

At the same time, the following logs were generated on Alfresco side:

2017-11-14 08:34:50,342  ERROR [aoservices-err.StandardWebdavService] [ajp-nio-8009-exec-7] doMove: BAD REQUEST: Destination malformed
2017-11-14 08:34:50,442  ERROR [aoservices-err.StandardWebdavService] [ajp-nio-8009-exec-8] doMove: BAD REQUEST: Destination malformed
2017-11-14 08:34:50,544  ERROR [aoservices-err.StandardWebdavService] [ajp-nio-8009-exec-9] doMove: BAD REQUEST: Destination malformed
2017-11-14 08:34:50,647  ERROR [aoservices-err.StandardWebdavService] [ajp-nio-8009-exec-1] doMove: BAD REQUEST: Destination malformed

With the default log level, that’s not particularly helpful… From the Apache logs, it is pretty easy to see when the folder “New Folder” has been created (MKCOL @ 08:34:21) as well as when I tried to rename it (MOVE @ 08:34:50):

alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:34:21 +0100] "MKCOL /alfresco/aos/Sites/my-site/documentLibrary/New%20folder HTTP/1.1" 201 640 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:34:21 +0100] "PROPPATCH /alfresco/aos/Sites/my-site/documentLibrary/New%20folder HTTP/1.1" 207 971 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:34:21 +0100] "PROPFIND /alfresco/aos/Sites/my-site/documentLibrary/New%20folder/desktop.ini HTTP/1.1" 404 588 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:34:50 +0100] "PROPFIND /alfresco/aos HTTP/1.1" 207 5473 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:34:50 +0100] "PROPFIND /alfresco/aos/Sites/my-site/documentLibrary HTTP/1.1" 207 1784 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:34:50 +0100] "PROPFIND /alfresco/aos/Sites/my-site/documentLibrary/New%20folder HTTP/1.1" 207 1803 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:34:50 +0100] "PROPFIND /alfresco/aos/Sites/my-site/documentLibrary HTTP/1.1" 207 1784 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:34:50 +0100] "MOVE /alfresco/aos/Sites/my-site/documentLibrary/New%20folder HTTP/1.1" 400 1606 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:34:50 +0100] "MOVE /alfresco/aos/Sites/my-site/documentLibrary/New%20folder HTTP/1.1" 400 1711 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:34:50 +0100] "MOVE /alfresco/aos/Sites/my-site/documentLibrary/New%20folder HTTP/1.1" 400 1711 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:34:50 +0100] "MOVE /alfresco/aos/Sites/my-site/documentLibrary/New%20folder HTTP/1.1" 400 1711 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:34:50 +0100] "PROPFIND /alfresco/aos/Sites/my-site/documentLibrary/New%20folder HTTP/1.1" 207 1909 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"

As you can see above, the renaming failed and the Apache responded with a 400 error code. The error “doMove: BAD REQUEST” on the Alfresco side reminded me of this JIRA but the outcome of this ticket was that the parameter “aos.baseUrlOverwrite” was wrongly set… In our case, its value was “https://alfresco_server_01.domain.com:443/alfresco/aos” (as shown above) and this seemed to be the correct URL… But in fact it wasn’t.

Just to avoid any uncertainty, we tried to change the value to “https://alfresco_server_01.domain.com/alfresco/aos” (so just removing the port :443 which technically can be here or not…) and then restart Alfresco… After doing that, the rename was actually working:

So magically the issue was gone… The associated Apache HTTPD logs showed this time a 201 return code:

alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:48:04 +0100] "PROPFIND /alfresco/aos HTTP/1.1" 207 5347 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:48:04 +0100] "PROPFIND /alfresco/aos/Sites/my-site/documentLibrary/New%20folder HTTP/1.1" 207 1799 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:48:05 +0100] "PROPFIND /alfresco/aos/Sites/my-site/documentLibrary HTTP/1.1" 207 1780 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:48:05 +0100] "PROPFIND /alfresco/aos/Sites HTTP/1.1" 207 1736 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:48:05 +0100] "PROPFIND /alfresco HTTP/1.1" 302 224 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:48:05 +0100] "PROPFIND /alfresco/ HTTP/1.1" 207 1858 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:48:05 +0100] "PROPFIND / HTTP/1.1" 302 572 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:48:05 +0100] "PROPFIND /share/page/repository HTTP/1.1" 501 1490 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:48:06 +0100] "MOVE /alfresco/aos/Sites/my-site/documentLibrary/New%20folder HTTP/1.1" 201 640 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"
alfresco_server_01.domain.com:443 10.20.30.40 - - [14/Nov/2017:08:48:07 +0100] "PROPFIND /alfresco/aos/Sites/my-site/documentLibrary/Renamed%202 HTTP/1.1" 207 1797 "-" "Microsoft-WebDAV-MiniRedir/6.3.9600"

Conclusion for this blog? Take care when you want to set the “aos.baseUrlOverwrite”, do not add the port if it is not really necessary! Another solution to this issue would be to just comment the “aos.baseUrlOverwrite” parameter since it is not needed when using Apache HTTPD. I personally never use this parameter (keeping it commented) because I’m always using Apache :).

Cet article Alfresco – Unable to move/rename a file/folder using AOS est apparu en premier sur Blog dbi services.

Here we are, the Alfresco DevCon 2018 day-1 is over (well except for the social party)! It’s been already 2 years I attended my last Alfresco event (BeeCon 2016, first of its name (organized by the Order of the Bee)) because I wasn’t able to attend the second BeeCon (2017) since it happened on the exact dates of our internal dbi xChange event. Yesterday was the DevCon 2018 day-0 with the Hack-a-thon, the full day training and the ACSCE/APSCE Certification preparation but today was really the first day of sessions.

The day-1 started, as usual, with a Keynote from Thomas DeMeo which presented interesting information regarding the global direction of Alfresco products, the Roadmap (of course) for the coming year as well as some use cases where Alfresco was successfully used in very interesting projects including also AWS.

The second part of the keynote has been presented by Brian Remmington which explained the future of the Alfresco Digital Platform. In the next coming months/years, Alfresco will include/refactor/work on the following points for its Digital Platform:

• Improve SSO solutions. Kerberos is already working very well with Alfresco but they intend to also add SAML2, OAuth, aso… This is a very good thing!
• Merging the Identity management for the ACS and APS into one single unit
• Adding an API Gateway in front of ACS and APS to always talk to the same component and targeting in the background both the ACS and APS. It will also allow Alfresco to change the backend APIs, if needed (to align them for example), without the API Gateway noticing it. This is a very good thing too from a developer perspective since you will be sure that your code will not break if Alfresco rename something for example!
• Merging the search on the ACS and APS into one single search index
• We already knew it but it was confirmed that Alfresco [will probably drop the default installer and instead] will provide docker/kubernetes means for you to deploy Alfresco easily and quickly using these new technologies
• Finishing the merge/refactor of other ACS/APS services into common units for both products so that work done once isn’t duplicated. This will concern the Search (Insight?) Service, the Transformation Service, the Form Service and a new Function Service (basically code functions shared between ACS and APS!).

All this looks promising, like really.

Then starting at 10am, there were four streams running in parallel so there is something that you will find interesting, that’s for sure. I didn’t mention it but DevCon isn’t just a name… It means that the sessions are really technical, we are far from the (boring) Business presentations that you can find on all other competitors’ events… I did a full morning on ADF. Mario Romano and Ole Hejlskov were presenting ADF Basics and Beyond.

For those of you who don’t know yet, ADF (Alfresco Development Framework) is the last piece of the Digital Platform that Alfresco has been bringing recently. It is a very interesting new framework that allows you to create your own UI to use in front of the ACS/APS. There are at the moment more than 100 angular components that you can use, extend, compose and configure to build the UI that will match your use case. Alfresco Share still provide way more features than ADF but I must say that I’m pretty impressed by what you can achieve in ADF with very little: it looks like it is going in the right direction.

ADF 2.0 has been released recently (November 2017) and it is based on three main pillars: the latest version of Angular 5, a powerful JavaScript API (that talk in the background with the ACS/APS/AGS APIs) and the Yeoman generator+Angular CLI for fast deployments.

ADF provides 3 extensions points for you to customize a component:

• html extension points => adding html to customize the look&feel or the behavior
• event listeners => adding behaviors on events for example
• config properties => each component has properties that will customize it

One of the goal of ADF is the ability to upgrade your application without any efforts. Angular components will be updated in the future but it was designed (and Alfresco effort is going) in a way that even if you use these components in your ADF application, then an upgrade of your application won’t hurt at all. If you want to lean more about ADF, then I suggest you the Alfresco Tech Talk Live that took place in December as well as the Alfresco Office Hours.

After this first introduction session to ADF, Eugenio Romano went deeper and showed how to play with ADF 2.0: installing it, deploying a first application and then customizing the main pieces like the theme, the document list, the actions, the search, aso… There were some really interesting examples and I’m really looking forward seeing the tutorials and documentations popping up on the Alfresco Website about these ADF 2.0 new features and components.

To conclude the morning, Denys Vuika presented a session about how to use and Extend the Alfresco Content App (ACA). The ACA is the new ADF 2.0 application provided by Alfresco. It is a demo/sample application whose purpose is to be lightweight so it is as fast as possible. You can then customize it as you want, play with the ADF so that this sample application match your needs. One of the demo Denys presented is how you can change the default previewer for certain type of files (.txt, .js, .xml for example). In ADF, that’s like 5 lines of code (of course you need to have another previewer of your own but that’s not ADF stuff) and then he had an awesome preview for .js files where there were syntax highlighting right inside the Alfresco preview as well as tooltips on names to give description of variables and functions apparently. This kind of small features but done so easily look quite promising.

I already wrote a lot on ADF today so I will stop my blog here but I did attend a lot of other very interesting sessions on the afternoon. I might talk about that tomorrow.

Cet article Alfresco DevCon 2018 – Day 1 – ADF, ADF and… ADF! est apparu en premier sur Blog dbi services.

Today is the second day of the Alfresco DevCon 2018 and therefore yes, it is already over, unfortunately. In this blog, I will be continuing my previous one with sessions I attended on the afternoon of the day-1 as well as day-2. There were too many interesting sessions and I don’t really have the time to talk about all of them… But if you are interested, all the sessions were recorded (as always) so wait a little bit and check out the DevCon website, the Alfresco Community or the Alfresco Youtube channel and I’m sure you will find all the recordings as soon as they are available.

So on the afternoon of the day-1, I started with a presentation of Jeff Potts, you all know him, and he was talking about how to move in (upload) and out (download) of Alfresco some gigantic files (several gigabytes). He basically presented a use case where the users had to manage big files and put them all in Alfresco with the less headache possible. On the paper, Alfresco can handle any file no matter the size because the only limit is what the File System of the Alfresco Server supports. However, when you start working with 10 or 20 GB files, you can sometimes have issues like exceptions, timeouts, network outage, aso… It might not be frequent but it can happen for a variety of reasons (not always linked to Alfresco). The use case here was to simplify the import into Alfresco and make it faster. Jeff tested a lot of possible solutions like using the Desktop Sync, CMIS, FTP, the resumable upload share add-on, aso…

In the end, a pure simple (1 stream) upload/download will always be limited by the network. So he tried to work on improving this part and used the Resilio Sync software (formerly BitTorrent Sync). This tool can be used to stream a file to the Alfresco Server, BitTorrent style (P2P). But the main problem of this solution is that P2P is only as good as the number of users having this specific file available on their workstation… Depending on the use case, it might increase the performance but it wasn’t ideal.

In the end, Jeff came across the protocol “GridFTP”. This is an extension of the FTP for grid computing whose purpose is to make the file transfer more reliable and faster using multiple simultaneous TCP streams. There are several implementations of the GridFTP like the Globus Toolkit. Basically, the solution in this case was to use Globus to transfer the big files from the user’s workstation to a dedicated File System which is mounted on the Alfresco Server. Then using the Alfresco Bulk FileSystem Import Tool (BFSIT), it is really fast to import documents into Alfresco, as soon as they are on the File System of the Alfresco Server. For the download, it is just the opposite (using the BFSET)…

For files smaller than 512Mb, this solution is probably slower than the default Alfresco upload/download actions but for bigger files (or group of files), then it becomes very interesting. Jeff did some tests and basically for one or several files with a total size of 3 or 4GB, then the transfer using Globus and then the import into Alfresco was 50 to 60% faster than the Alfresco default upload/download.

Later, Jose Portillo shared Solr Sharding Best Practices. Sharding is the action of splitting your indexes into Shards (part of an index) to increase the searches and indexing (horizontal scaling). The Shards can be stored on a single Solr Server or they can be dispatched on several. Doing this basically increase the search speed because the search is executed on all Shards. For the indexing of a single node, there is no big difference but for a full reindex, it does increase a lot the performance because you do index several nodes at the same time on each Shards…

A single Shard can work well (according to the Alfresco Benchmark) with up to 50M documents. Therefore, using Shards is mainly for big repositories but it doesn’t mean that there are no use cases where it would be interesting for smaller repositories, there are! If you want to increase your search/index performance, then start creating Shards much earlier.

For the Solr Sharding, there are two registration options:

• Manual Sharding => You need to manually configure the IPs/Host where the Shards are located in the Alfresco properties files
• Dynamic Sharding => Easier to setup and Alfresco automatically provide information regarding the Shards on the Admin interface for easy management

There are several methods of Shardings which are summarized here:

• MOD_ACL_ID (ACL v1) => Sharding based on ACL. If all documents have the same ACL (same site for example), then they will all be on the same Shard, which might not be very useful…
• ACL_ID (ACL v2) => Same as v1 except that it uses the murmur hash of the ACL ID and not its modulus
• DB_ID (DB ID) => Default in Solr6. Nodes are evenly distributed on the Shards based on their DB ID
• DB_ID_RANGE (DB ID Range) => You can define the DB ID range for which nodes will go to which Shard (E.g.: 1 to 10M => Shard-0 / 10M to 20M => Shard-1 / aso…)
• DATE (Date and Time) => Assign date for each Shards based on the month. It is possible to group some months together and assign a group per Shard
• PROPERTY (Metadata) => The value of some property is hashed and this hash is used for the assignment to a Shard so all nodes with the same value are in the same Shard
• EXPLICIT_ID (Explicit) => This is an all-new method that isn’t yet on the documentation… It is apparently working similarly to the Metadata one but it’s not hashing the value of the property but using it directly/explicitly to know which shard should be used to index this document. If the property is missing or invalid, it should apparently fall back to the DB_ID method

Unfortunately, the Solr Sharding has only been available starting with Alfresco Content Services 5.1 (Solr 4) and only using the ACL v1 method. New methods were then added using the Alfresco Search Services (Solr 6). The availability of methods VS Alfresco/Solr versions has been summarized in Jose’s presentation:

Jose also shared a comparison matrix of the different methods to choose the right one for each use case:

Some other best practices regarding the Solr Sharding:

• Replicate the Shards to increased response time and it also provides High Availability so… No reasons not to!
• Backup the Shards using the provided Web Service so Alfresco can do it for you for one or several Shards
• Use DB_ID_RANGE if you want to be able to add Shards without having to perform a full reindex, this is the only way
• If you need another method than DB_ID_RANGE, then plan carefully the number of Shards to be created. You might want to overshard to take into account the future growth
• Keep in mind that each Shard will pull the changes from Alfresco every 15s and it all goes to the DB… It might create some load there and therefore be sure that your DB can handle that
• As far as I know, at the moment, the Sharding does not support Solr in SSL. Solr should anyway be protected from external accesses because it is only used by Alfresco internally so this is an ugly point so far but it’s not too bad. Sharding is pretty new so it will probably support the SSL at some point in the future
• Tune properly Solr and don’t forget the Application Server request header size
  • Solr4 => Tomcat => maxHttpHeaderSize=…
  • Solr6 => Jetty => solr.jetty.request.header.size=…

The day-2 started with a session from John Newton which presented the impact of emerging technologies on content. As usual, John’s presentation had a funny theme incorporated in the slides and this time it was Star Wars.

After that, I attended the Hack-a-thon showcase, presented/introduced by Axel Faust. In the Alfresco world, Hack-a-thons are:

• There since 2012
• Open-minded and all about collaboration. Therefore, the output of any project is open source and available for the community. It’s not about money!
• Always the source of great add-ons and ideas
• 2 times per year
  • During conferences (day-0)
  • Virtual Hack-a-thon (36h ‘follow-the-sun’ principle)

A few of the 16 teams that participated in the Hack-a-thon presented the result of their Hack-a-thon day and there were really interesting results for ACS, ACS on AWS, APS, aso…

Except that, I also attended all lightning talks on this day-2 as well as presentations on PostgreSQL and Solr HA/Backup solutions and best practices. The presentations about PostgreSQL and Solr were interesting especially for newcomers because it really explained what should be done to have a highly available and resilient Alfresco environment.

There were too many lightning talk to mention them all but as always, there were some quite interesting and there I just need to mention the talk about the ContentCraft plugin (from Roy Wetherall). There cannot be an Alfresco event (be it a Virtual Hack-a-thon, BeeCon or DevCon now) without an Alfresco integration into Minecraft. Every year, Roy keeps adding new stuff into his plugin… I remember years ago, Roy was already able to create a building in Minecraft where the height represented the number of folders stored in Alfresco and the depth was the number of documents inside, if my memory is correct (this changed now, it represents the number of sub-folders). This year, Roy presented the new version and it’s even more incredible! Now if you are in front of one of the building’s door, you can see the name and creator of the folder in a ‘Minecraft sign’. Then you can walk in the building and there is a corridor. On both sides, there are rooms which represent the sub-folders. Again, there are ‘Minecraft signs’ there with the name and creator of the sub-folders. Until then, it’s just the same thing again so that’s cool but it will get even better!

If you walk in a room, you will see ‘Minecraft bookshelves’ and ‘Minecraft chests’. Bookshelves are just there for the decoration but if you open the chests, then you will see, represented by ‘Minecraft books’, all your Alfresco documents stored on this sub-folder! Then if you open a book, you will see the content of this Alfresco document! And even crazier, if you update the content of the book on Minecraft and save it, the document stored in Alfresco will reflect this change! This is way too funny :D.

It’s all done using CMIS so there is nothing magical… Yet it really makes you wonder if there are any limits to what Alfresco can do ;).

If I dare to say: long live Alfresco! And see you around again for the next DevCon.

Cet article Alfresco DevCon 2018 – Day 2 – Big files, Solr Sharding and Minecraft, again! est apparu en premier sur Blog dbi services.

ActiveMQ is an open source Java Messaging Server (JMS) from the Apache Software Foundation that supports a lot of protocols. In Alfresco 5, ActiveMQ has been introduced as a new, optional, component in the stack. It was, at the beginning, only used for “side” features like Alfresco Analytics or Alfresco Media Management in the early Alfresco 5.0. In Alfresco 6.0, ActiveMQ was still used for Alfresco Media Management but also for the Alfresco Sync Service. It’s only starting with the Alfresco 6.1, released last February, that ActiveMQ became a required component, used for the same things but also now used for transformations.

The Alfresco documentation doesn’t really describe how to install ActiveMQ or how to configure it, it just explains how to connect Alfresco to it. Therefore, I thought I would write a small blog about how to do a basic installation of ActiveMQ for a usage in Alfresco.

Alfresco 6.1 supports ActiveMQ v5.15.6 so that’s the one I will be using for this blog as example.

First let’s start with defining some environment variables that will be used to know where to put ActiveMQ binaries and data:

[alfresco@mq_n1 ~]$ echo "export ACTIVEMQ_HOME=/opt/activemq" >> ~/.profile
[alfresco@mq_n1 ~]$ echo "export ACTIVEMQ_DATA=\$ACTIVEMQ_HOME/data" >> ~/.profile
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ grep "ACTIVEMQ" ~/.profile
export ACTIVEMQ_HOME=/opt/activemq
export ACTIVEMQ_DATA=$ACTIVEMQ_HOME/data
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ source ~/.profile
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ echo $ACTIVEMQ_DATA
/opt/activemq/data
[alfresco@mq_n1 ~]$

I’m usually using symlinks for all the components so that I can keep a generic path in case of upgrades, aso… So, let’s download the software and put all that where it should:

[alfresco@mq_n1 ~]$ activemq_version="5.15.6"
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ wget http://archive.apache.org/dist/activemq/${activemq_version}/apache-activemq-${activemq_version}-bin.tar.gz
--2019-07-25 16:55:23--  http://archive.apache.org/dist/activemq/5.15.6/apache-activemq-5.15.6-bin.tar.gz
Resolving archive.apache.org... 163.172.17.199
Connecting to archive.apache.org|163.172.17.199|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 58556801 (56M) [application/x-gzip]
Saving to: ‘apache-activemq-5.15.6-bin.tar.gz’

apache-activemq-5.15.6-bin.tar.gz     100%[=======================================================================>]  55.84M  1.62MB/s    in 35s

2019-07-25 16:55:58 (1.60 MB/s) - ‘apache-activemq-5.15.6-bin.tar.gz’ saved [58556801/58556801]

[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ tar -xzf apache-activemq-${activemq_version}-bin.tar.gz
[alfresco@mq_n1 ~]$ mkdir -p $ACTIVEMQ_HOME-${activemq_version}
[alfresco@mq_n1 ~]$ ln -s $ACTIVEMQ_HOME-${activemq_version} $ACTIVEMQ_HOME
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ ls -l $ACTIVEMQ_HOME/.. | grep -i activemq
lrwxr-xr-x   1 alfresco  alfresco        31 Jul 25 17:04 activemq -> /opt/activemq-5.15.6
drwxr-xr-x   2 alfresco  alfresco        64 Jul 25 17:03 activemq-5.15.6
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ rm -rf ./apache-activemq-${activemq_version}/data
[alfresco@mq_n1 ~]$ mkdir -p $ACTIVEMQ_DATA
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ mv apache-activemq-${activemq_version}/* $ACTIVEMQ_HOME/

Once that is done and before starting ActiveMQ for the first time, there are still some configurations to be done. It is technically possible to add a specific authentication for communications between Alfresco and ActiveMQ or setup the communications in SSL for example. It depends on the usage you will have for the ActiveMQ but as a minimal configuration for use with Alfresco, I believe that the default users (“guest” to access docbroker & “user” to access web console) should at least be removed and the admin password changed:

[alfresco@mq_n1 ~]$ activemq_admin_pwd="Act1v3MQ_pwd"
[alfresco@mq_n1 ~]$ activemq_broker_name="`hostname -s`"
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ # Remove user "user" from the web console
[alfresco@mq_n1 ~]$ sed -i "/^user:[[:space:]]*.*/d" $ACTIVEMQ_HOME/conf/jetty-realm.properties
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ # Remove user "guest" from the broker
[alfresco@mq_n1 ~]$ sed -i "/^guest.*/d" $ACTIVEMQ_HOME/conf/credentials.properties
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ # Change admin password
[alfresco@mq_n1 ~]$ sed -i "s/^admin=.*/admin=${activemq_admin_pwd}\n/" $ACTIVEMQ_HOME/conf/users.properties
[alfresco@mq_n1 ~]$ sed -i "s/^admin.*/admin: ${activemq_admin_pwd}, admin/" $ACTIVEMQ_HOME/conf/jetty-realm.properties
[alfresco@mq_n1 ~]$ sed -i "s/^activemq.username=.*/activemq.username=admin/" $ACTIVEMQ_HOME/conf/credentials.properties
[alfresco@mq_n1 ~]$ sed -i "s/^activemq.password=.*/activemq.password=${activemq_admin_pwd}/" $ACTIVEMQ_HOME/conf/credentials.properties
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ grep -E "brokerName|storeUsage |tempUsage " $ACTIVEMQ_HOME/conf/activemq.xml
    <broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="${activemq.data}">
                <storeUsage limit="100 gb"/>
                <tempUsage limit="50 gb"/>
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ # Set broker name & allowed usage
[alfresco@mq_n1 ~]$ sed -i "s/brokerName=\"[^\"]*\"/brokerName=\"${activemq_broker_name}\"/" $ACTIVEMQ_HOME/conf/activemq.xml
[alfresco@mq_n1 ~]$ sed -i 's,storeUsage limit="[^"]*",storeUsage limit="10 gb",' $ACTIVEMQ_HOME/conf/activemq.xml
[alfresco@mq_n1 ~]$ sed -i 's,tempUsage limit="[^"]*",tempUsage limit="5 gb",' $ACTIVEMQ_HOME/conf/activemq.xml
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ grep -E "brokerName|storeUsage |tempUsage " $ACTIVEMQ_HOME/conf/activemq.xml
    <broker xmlns="http://activemq.apache.org/schema/core" brokerName="mq_n1" dataDirectory="${activemq.data}">
                    <storeUsage limit="10 gb"/>
                    <tempUsage limit="5 gb"/>
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ chmod -R o-rwx $ACTIVEMQ_HOME
[alfresco@mq_n1 ~]$ chmod -R o-rwx $ACTIVEMQ_DATA

So above, I set a specific name for the broker, that’s mainly if you expect to see at some points several brokers, to differentiate them. I also change the default storeUsage and tempUsage, that’s mainly to show how it’s done because these two parameters define the limit that ActiveMQ will be able to use on the file system. I believe the default is way too much for ActiveMQ’s usage in Alfresco, so I always reduce these or use a percentage as value (percentLimit).

With the default configuration, ActiveMQ uses “${activemq.data}” for the data directory which is actually using the “$ACTIVEMQ_DATA” environment variable, if present (otherwise it sets it as $ACTIVEMQ_HOME/data). That’s the reason why I set this environment variable, so it is possible to define a different data folder without having to change the default configuration. This data folder will mainly contain the logs of ActiveMQ, the PID file and the KahaDB for the persistence adapter.

Finally creating a service for ActiveMQ and starting it is pretty easy as well:

[alfresco@mq_n1 ~]$ cat > activemq.service << EOF
[Unit]
Description=ActiveMQ service

[Service]
Type=forking
ExecStart=###ACTIVEMQ_HOME###/bin/activemq start
ExecStop=###ACTIVEMQ_HOME###/bin/activemq stop
Restart=always
User=alfresco
WorkingDirectory=###ACTIVEMQ_DATA###
LimitNOFILE=8192:65536

[Install]
WantedBy=multi-user.target
EOF
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ sed -i "s,###ACTIVEMQ_HOME###,${ACTIVEMQ_HOME}," activemq.service
[alfresco@mq_n1 ~]$ sed -i "s,###ACTIVEMQ_DATA###,${ACTIVEMQ_DATA}," activemq.service
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ sudo cp activemq.service /etc/systemd/system/
[alfresco@mq_n1 ~]$ rm activemq.service
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ sudo systemctl enable activemq.service
[alfresco@mq_n1 ~]$ sudo systemctl daemon-reload
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ sudo systemctl start activemq.service

Once ActiveMQ is setup as you want, for the registration in Alfresco, it’s very easy:

[alfresco@alf_n1 ~]$ cat $CATALINA_HOME/shared/classes/alfresco-global.properties
...
### ActiveMQ
messaging.broker.url=failover:(tcp://mq_n1.domain:61616?daemon=false&dynamicManagement=false&trace=false)?timeout=3000&randomize=false
#messaging.username=
#messaging.password=
...
[alfresco@alf_n1 ~]$

As mentioned at the beginning of this blog, ActiveMQ supports a lot of protocols so you can use pretty much what you want: TCP, NIO, SSL, NIO SSL, Peer (2 Peer), UDP, Multicast, HTTP, HTTPS, aso… You can find all the details for that here.

To add authentication between Alfresco and ActiveMQ, you will need to enable the properties in the alfresco-global.properties (the two that I commented above) and define the appropriate authentication in the ActiveMQ broker configuration. There is an example in the Alfresco documentation.

Cet article Alfresco – ActiveMQ basic setup est apparu en premier sur Blog dbi services.

This blog will be the first of a series on Alfresco HA/Clustering topics. It’s been too long I haven’t posted anything related to Alfresco so I thought about writing a few blogs about my experience with setting up more or less complex HA/Clustering infrastructures. So, let’s start this first part with an introduction to the Alfresco HA/Clustering.

If you want to setup a HA/Cluster environment, you will have to first think about where you want to go exactly. Alfresco is composed of several components so “what do you want to achieve exactly?”, that would probably be the first question to ask.

Alfresco offers a lot of possibilities, you can more or less do whatever you want. That’s really great, but it also means that you should plan what you want to do first. Do you just want a simple HA architecture for Share+Repository but you can live without Solr for a few minutes/hours (in case of issues) or you absolutely want all components to be always available? Or maybe you want an HA architecture which is better suited for high throughput? Obviously, there might be some costs details that need to be taken into consideration linked to the resources but also the licenses: the Alfresco Clustering license itself but also the Index Engine license if you go for separated Solr Servers.

That’s what you need to define first to avoid losing time changing configurations and adding more components into the picture later. Alternatively (and that’s something I will try to cover as much as I can), it’s also possible to setup an environment which will allow you to add more components (at least some of them…) as needed without having to change your HA/Clustering configuration, if you are doing it right from the start and if you don’t change too much the architecture itself.

I mentioned earlier the components of Alfresco (Alfresco Content Services, not the company), these are the ones we are usually talking about:

• *ActiveMQ
• Alfresco PDF Renderer
• Database
• File System
• *Front-end (Apache HTTPD, Nginx, …)
• ImageMagick
• Java
• LibreOffice
• *Share (Tomcat)
• *Solr6 (Jetty)
• *Repository (Tomcat)

In this series of blog, I won’t talk about the Alfresco PDF Renderer, ImageMagick & Java because these are just simple binaries/executables that need to be available from the Repository side. For LibreOffice, it’s usually Alfresco that is managing it directly (multi-processes, restart if crash, aso…). It wouldn’t really make sense to talk about these in blogs related to Clustering. I will also disregard the Database and File System ones since they are usually out of my scope. The Database is usually installed & managed by my colleagues which are DBAs, they are much better at that than me. That leaves us with all components with an asterisk (*). I will update this list with links to the different blogs.

Before jumping in the first component, which will be the subject of the next blog, I wanted to go through some possible architectures for Alfresco. There are a lot of schemas available on internet but it’s often the same architecture that is presented so I thought I would take some time to represent, in my own way, what the Alfresco’s architecture could look like.

In the below schemas, I represented the main components: Front-end, Share, Repository, Solr, Database & File System (Data) as little boxes. As mentioned previously, I won’t talk about the Database & File System so I just represented them once to see the communications with these but what is behind their boxes can be anything (with HA/Clustering or not). The arrows represent the way communications are initiated: an arrow in a single direction “A->B” means that B is never initiating a communication with A. Boxes that are glued together represent all components installed on the same host (a physical server, a VM, a container or whatever).

N°1: This is the simplest architecture for Alfresco. As you can see, it’s not a HA/Clustering architecture but I decided to start small. I added a Front-end (even if it’s not mandatory) because it’s a best practice and I would not install Alfresco without it. Nothing specific to say on this architecture, it’s just simple.

N°2: The first thing to do if you have the simplest architecture in place (N°1) and you start seeing some resources contention is to split the components and more specifically to install Solr separately. This should really be the minimal architecture to use, whenever possible.

N°3: This is the first HA/Clustering architecture. It starts small as you can see with just two nodes for each Front-end/Share/Repository stack with a Load Balancer to dispatch the load on each side for an Active/Active solution. The dotted grey lines represent the Clustering communications. In this architecture, there is therefore a Clustering for Share and another one for the Repository layer. The Front-end doesn’t need Clustering since it just forwards the communications but the session itself is on the Tomcat (Share/Repository) side. There is only one Solr node and therefore both Repository boxes will communicate with the Solr node (through the Front-end or not). Between the Repository and Solr, there is one bidirectional arrow and another one unidirectional. That’s because both Repository boxes will initiate searches but the Solr will do tracking to index new content with only one Repository: this isn’t optimal.

N°4: To solve this small issue with Solr tracking, we can add a second Load Balancer in between so that the Solr tracking can target any Repository node. The first bottleneck you will encounter in Alfresco is usually the Repository because a lot of things are happening in the background at that layer. Therefore, this architecture is usually the simplest HA/Clustering solution that you will want to setup.

N°5: If you are facing some performance issues with Solr or if you want all components to be in HA, then you will have to duplicate the Solr as well. Between the two Solr nodes, I put a Clustering link, that’s in case you are using Solr Sharding. If you are using the default cores (alfresco and archive), then there is no communication between distinct Solr nodes. If you are using Solr Sharding and if you want a HA architecture, then you will have the same shards on both Solr nodes and in this case, there will be communications between the Solr nodes, it’s not really a Clustering so to speak, that’s how Solr Sharding is working but I still used the same representation.

N°6: As mentioned previously (for the N°4), the Repository is usually the bottleneck. To reduce the load on this layer, it is possible to do several things. The first possibility is to install another Repository and dedicate it to the Solr Tracking. As you can see above, the communications aren’t bidirectional anymore but only unidirectional. Searches will come from the two Repository that are in Cluster and Solr Tracking will use the separated/dedicated Repository. This third Repository can then be set in read-only, the jobs and services can be disabled, the Clustering can be disabled as well (so it uses the same DB but it’s not part of the Clustering communications because it doesn’t have to), aso… I put this third Repository as a standalone box but obviously you can install it with one of the two Solr nodes.

N°7: The next step can be to add another read-only Repository and put these two nodes side by side with the Solr nodes. This is to only have localhost communications for the Solr Tracking which is therefore a little bit easier to secure.

N°8: The previous architectures (N°6 & N°7) introduced a new single point of failure so to fix this, there is only one way: add a new Load Balancer between the Solr and the Repository for the tracking. Behind the Load Balancer, there are two solutions: keep the fourth Repository which is also in read-only or use a fallback to the Repository node1/node2 in case the read-only Repository (node3) isn’t available. For that purpose, the Load Balancer should be in, respectively, Active/Active or Active/Passive. As you can see, I choose to represent the first one.

These were a few possible architectures. You can obviously add more nodes if you want to, to handle more load. There are many other solutions so have fun designing the best one, according to your requirements.

Cet article Alfresco Clustering – Basis & Architectures est apparu en premier sur Blog dbi services.

In a previous blog, I talked about some basis and presented some possible architectures for Alfresco. Now that this introduction has been done, let’s dig into the real blogs about how to setup a HA/Clustering Alfresco environment. In this blog in particular, I will talk about the Repository layer.

For the Repository Clustering, there are three prerequisites (and that’s all you need):

• A valid license which include the Repository Clustering
• A shared file system which is accessible from all Alfresco nodes in the Cluster. This is usually a NAS accessed via NFS
• A shared database

Clustering the Repository part is really simple to do: you just need to put the correct properties in the alfresco-global.properties file. Of course, you could also manage it all from the Alfresco Admin Console but that’s not recommended, you should really always use the alfresco-global.properties by default. The Alfresco Repository Clustering is using Hazelcast. It was using JGroups and EHCache as well before Alfresco 4.2 but now it’s just Hazelcast. So to define an Alfresco Cluster, simply put the following configuration in the alfresco-global.properties of the Alfresco node1:

[alfresco@alf_n1 ~]$ getent hosts `hostname` | awk '{ print $1 }'
10.10.10.10
[alfresco@alf_n1 ~]$
[alfresco@alf_n1 ~]$ cat $CATALINA_HOME/shared/classes/alfresco-global.properties
...
### Content Store
dir.root=/shared_storage/alf_data
...
### DB
db.username=alfresco
db.password=My+P4ssw0rd
db.name=alfresco
db.host=db_vip
## MySQL
#db.port=3306
#db.driver=com.mysql.jdbc.Driver
#db.url=jdbc:mysql://${db.host}:${db.port}/${db.name}?useUnicode=yes&characterEncoding=UTF-8
#db.pool.validate.query=SELECT 1
## PostgreSQL
db.driver=org.postgresql.Driver
db.port=5432
db.url=jdbc:postgresql://${db.host}:${db.port}/${db.name}
db.pool.validate.query=SELECT 1
## Oracle
#db.driver=oracle.jdbc.OracleDriver
#db.port=1521
#db.url=jdbc:oracle:thin:@${db.host}:${db.port}:${db.name}
#db.pool.validate.query=SELECT 1 FROM DUAL
...
### Clustering
alfresco.cluster.enabled=true
alfresco.cluster.interface=10.10.10.10-11
alfresco.cluster.nodetype=Alfresco_node1
alfresco.hazelcast.password=Alfr3sc0_hz_Test_pwd
alfresco.hazelcast.port=5701
alfresco.hazelcast.autoinc.port=false
alfresco.hazelcast.max.no.heartbeat.seconds=15
...
[alfresco@alf_n1 ~]$

And for the Alfresco node2, you can use the same content:

[alfresco@alf_n2 ~]$ getent hosts `hostname` | awk '{ print $1 }'
10.10.10.11
[alfresco@alf_n2 ~]$
[alfresco@alf_n2 ~]$ cat $CATALINA_HOME/shared/classes/alfresco-global.properties
...
### Content Store
dir.root=/shared_storage/alf_data
...
### DB
db.username=alfresco
db.password=My+P4ssw0rd
db.name=alfresco
db.host=db_vip
## MySQL
#db.port=3306
#db.driver=com.mysql.jdbc.Driver
#db.url=jdbc:mysql://${db.host}:${db.port}/${db.name}?useUnicode=yes&characterEncoding=UTF-8
#db.pool.validate.query=SELECT 1
## PostgreSQL
db.driver=org.postgresql.Driver
db.port=5432
db.url=jdbc:postgresql://${db.host}:${db.port}/${db.name}
db.pool.validate.query=SELECT 1
## Oracle
#db.driver=oracle.jdbc.OracleDriver
#db.port=1521
#db.url=jdbc:oracle:thin:@${db.host}:${db.port}:${db.name}
#db.pool.validate.query=SELECT 1 FROM DUAL
...
### Clustering
alfresco.cluster.enabled=true
alfresco.cluster.interface=10.10.10.10-11
alfresco.cluster.nodetype=Alfresco_node2
alfresco.hazelcast.password=Alfr3sc0_hz_Test_pwd
alfresco.hazelcast.port=5701
alfresco.hazelcast.autoinc.port=false
alfresco.hazelcast.max.no.heartbeat.seconds=15
...
[alfresco@alf_n2 ~]$

Description of the Clustering parameters:

• alfresco.cluster.enabled: Whether or not you want to enable the Repository Clustering for the local Alfresco node. The default value is false. You will want to set that to true for all Repository nodes that will be used by Share or any other client. If the Repository is only used for Solr Tracking, you can leave that to false
• alfresco.cluster.interface: This is the network interface on which Hazelcast will listen for Clustering messages. This has to be an IP, it can’t be a hostname. To keep things simple and to have the same alfresco-global.properties on all Alfresco nodes however, it is possible to use a specific nomenclature:
  • 10.10.10.10: Hazelcast will try to bind on 10.10.10.10 only. If it’s not available, then it won’t start
  • 10.10.10.10-11: Hazelcast will try to bind on any IP within the range 10-11 so in this case 2 IPs: 10.10.10.10 or 10.10.10.11. If you have, let’s say, 4 IPs assigned to the local host and you don’t want Hazelcast to use 2 of these, then specify the ones that it can use and it will pick one from the list. This can also be used to have the same content for the alfresco-global.properties on different hosts… One server with IP 10.10.10.10 and a second one with IP 10.10.10.11
  • 10.10.10.* or 10.10.*.*: Hazelcast will try to bind on any IP in this range, this is an extended version of the XX-YY range above
• alfresco.cluster.nodetype: A human-friendly string to represent the local Alfresco node. It doesn’t have any use for Alfresco, that’s really more for you. It is for example interesting to put a specific string for Alfresco node that won’t take part in the Clustering but that are still using the same Content Store and Database (like a Repository dedicated for the Solr Tracking, as mentioned above)
• alfresco.hazelcast.password: The password to use for the Alfresco Repository Cluster. You need to use the same password for all members of the same Cluster. You should as well try to use a different password for each Cluster that you might have if they are in the same network (DEV/TEST/PROD for example), otherwise it will get ugly
• alfresco.hazelcast.port: The default port that will be used for Clustering messages between the different members of the Cluster
• alfresco.hazelcast.autoinc.port: Whether or not you want to allow Hazelcast to find another free port in case the default port (“alfresco.hazelcast.port”) is currently used. It will increment the port by 1 each time. You should really set this to false and just use the default port, to have full control over the channels that Clustering communications are using otherwise it might get messy as well
• alfresco.hazelcast.max.no.heartbeat.seconds: The maximum time in seconds allowed between two heartbeat. If there is no heartbeat in this period of time, Alfresco will assume the remote node isn’t running/available

As you can see above, it’s really simple to add Clustering to an Alfresco Repository. Since you can(should?) have the same set of properties (except the nodetype string maybe), then it also really simplifies the deployment… If you are familiar with other Document Management System like Documentum for example, then you understand the complexity of some of these solutions! If you compare that to Alfresco, it’s like walking on the street versus walking on the moon where you obviously first need to go to the moon… Anyway, once it’s done, the logs of the Alfresco Repository node1 will display something like that when you start it:

2019-07-20 15:14:25,401  INFO  [cluster.core.ClusteringBootstrap] [localhost-startStop-1] Cluster started, name: MainRepository-<generated_id>
2019-07-20 15:14:25,405  INFO  [cluster.core.ClusteringBootstrap] [localhost-startStop-1] Current cluster members:
  10.10.10.10:5701 (hostname: alf_n1)

Wait for the Repository node1 to be fully started and once done, you can start the Repository node2, it needs to be started sequentially normally. You will see on the logs of the Repository node1 that another node joined automatically the Cluster:

2019-07-20 15:15:06,528  INFO  [cluster.core.MembershipChangeLogger] [hz._hzInstance_1_MainRepository-<generated_id>.event-3] Member joined: 10.10.10.11:5701 (hostname: alf_n2)
2019-07-20 15:15:06,529  INFO  [cluster.core.MembershipChangeLogger] [hz._hzInstance_1_MainRepository-<generated_id>.event-3] Current cluster members:
  10.10.10.10:5701 (hostname: alf_n1)
  10.10.10.11:5701 (hostname: alf_n2)

On the logs of the Repository node2, you can see directly at the initialization of the Hazelcast Cluster that the two nodes are available.

If you don’t want to check the logs, you can see pretty much the same thing from the Alfresco Admin Console. By accessing “http(s)://<hostname>:<port>/alfresco/s/enterprise/admin/admin-clustering“, you can see currently available cluster members (online nodes), non-available cluster members (offline nodes) as well as connected non-cluster members (nodes using the same DB & Content Store but with “alfresco.cluster.enabled=false”, for example to dedicate a Repository to Solr Tracking).

Alfresco also provides a small utility to check the health of the cluster which will basically ensure that the communication between each member is successful. This utility can be accessed at “http(s)://<hostname>:<port>/alfresco/s/enterprise/admin/admin-clustering-test“. It is useful to include a quick check using this utility in a monitoring solution for example, to ensure that the cluster is healthy.

Other posts of this series on Alfresco HA/Clustering:

• Basis & Architectures
• ActiveMQ
• Apache HTTPD as Load Balancer
• Share
• Solr6
• Repository

Cet article Alfresco Clustering – Repository est apparu en premier sur Blog dbi services.

In previous blogs, I talked about some basis and presented some possible architectures for Alfresco and I talked about the Clustering setup for the Alfresco Repository. In this one, I will work on the Alfresco Share layer. Therefore, if you are using another client like a CMIS/REST client or an ADF Application, it won’t work that way, but you might or might not need Clustering at that layer, it depends how the Application is working.

The Alfresco Share Clustering is used only for the caches, so you could technically have multiple Share nodes working with a single Repository or a Repository Cluster without the Share Clustering. For that, you could disable the caches on the Share layer because if you kept it enabled, you would have, eventually, faced issues. Alfresco introduced a Share Clustering which is used to keep the caches in sync, so you don’t have to disable it anymore. When needed, cache invalidation messages are sent from one Share node to all others, that include runtime application properties changes as well as new/existing site/user dashboards changes.

Just like for the Repository part, it’s really easy to setup the Share Clustering so there is really no reasons not to. It’s also using Hazelcast but it’s not based on properties that you need to configure in the alfresco-global.properties (because it’s a Share configuration), this one must be done in an XML file and there is no possibilities to do that in the Alfresco Admin Console, obviously.

All Share configuration/customization are put in the “$CATALINA_HOME/shared/classes/alfresco/web-extension” folder, this one is no exception. There are two possibilities for the Share Clustering communications:

• Multicast
• Unicast (TCP-IP in Hazelcast)

I. Multicast

If you do not know how many nodes will participate in your Share Cluster or if you want to be able to add more nodes in the future without having to change the previous nodes’ configuration, then you probably want to check and opt for the Multicast option. Just create a new file “$CATALINA_HOME/shared/classes/alfresco/web-extension/custom-slingshot-application-context.xml” and put this content inside it:

[alfresco@share_n1 ~]$ cat $CATALINA_HOME/shared/classes/alfresco/web-extension/custom-slingshot-application-context.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:hz="http://www.hazelcast.com/schema/spring"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
                           http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
                           http://www.hazelcast.com/schema/spring
                           http://www.hazelcast.com/schema/spring/hazelcast-spring-2.4.xsd">

  <hz:topic id="topic" instance-ref="webframework.cluster.slingshot" name="share_hz_test"/>
  <hz:hazelcast id="webframework.cluster.slingshot">
    <hz:config>
      <hz:group name="slingshot" password="Sh4r3_hz_Test_pwd"/>
      <hz:network port="5801" port-auto-increment="false">
        <hz:join>
          <hz:multicast enabled="true" multicast-group="224.2.2.5" multicast-port="54327"/>
          <hz:tcp-ip enabled="false">
            <hz:members></hz:members>
          </hz:tcp-ip>
        </hz:join>
        <hz:interfaces enabled="false">
          <hz:interface></hz:interface>
        </hz:interfaces>
      </hz:network>
    </hz:config>
  </hz:hazelcast>

  <bean id="webframework.cluster.clusterservice" class="org.alfresco.web.site.ClusterTopicService" init-method="init">
    <property name="hazelcastInstance" ref="webframework.cluster.slingshot" />
    <property name="hazelcastTopicName">
      <value>share_hz_test</value>
    </property>
  </bean>

</beans>
[alfresco@share_n1 ~]$

In the above configuration, be sure to set a topic name (matching the hazelcastTopicName’s value) as well as a group password that is specific to this environment, so you don’t end-up with a single Cluster with members coming from different environments. For the Share layer, it’s less of an issue than for the Repository layer but still. Be sure also to use a network port that isn’t in use, it will be the port that Hazelcast will bind itself to in the local host. For Alfresco Clustering, we used 5701 so here it’s 5801 for example.

Not much more to say about this configuration, we just enabled the multicast with an IP and a port to be used and we disabled the tcp-ip one.

The interfaces is disabled by default but you can enable it, if you want to. If it’s disabled, Hazelcast will list all local interfaces (127.0.0.1, local_IP1, local_IP2, …) and it will choose one in this list. If you want to force Hazelcast to use a specific local network interface, then enable this section and add that here. In can use the following nomenclature (IP only!):

• 10.10.10.10: Hazelcast will try to bind on 10.10.10.10 only. If it’s not available, then it won’t start
• 10.10.10.10-11: Hazelcast will try to bind on any IP within the range 10-11 so in this case 2 IPs: 10.10.10.10 or 10.10.10.11. If you have, let’s say, 5 IPs assigned to the local host and you don’t want Hazelcast to use 3 of these, then specify the ones that it can use and it will pick one from the list. This can also be used to have the same content for the custom-slingshot-application-context.xml on different hosts… One server with IP 10.10.10.10 and a second one with IP 10.10.10.11
• 10.10.10.* or 10.10.*.*: Hazelcast will try to bind on any IP in this range, this is an extended version of the XX-YY range above

For most cases, keeping the interfaces disabled is sufficient since it will just pick one available. You might think that Hazelcast may bind itself to 127.0.0.1, technically it’s possible since it’s a local network interface but I have never seen it do so, so I assume that there is some kind of preferred order if another IP is available.

Membership in Hazelcast is based on “age”, meaning that the oldest member will be the one to lead. There is no predefined Master or Slave members, they are all equal, but the oldest/first member is the one that will check if new members are allowed to join (correct config) and if so, it will send the information to all other members that joined already so they are all aligned. If multicast is enabled, a multicast listener is started to listen for new membership requests.

II. Unicast

If you already know how many nodes will participate in your Share Cluster or if you prefer to avoid Multicast messages (there is no real need to overload your network with such things…), then it’s preferable to use Unicast messaging. For that purpose, just create the same file as above (“$CATALINA_HOME/shared/classes/alfresco/web-extension/custom-slingshot-application-context.xml“) but instead, use the tcp-ip section:

[alfresco@share_n1 ~]$ cat $CATALINA_HOME/shared/classes/alfresco/web-extension/custom-slingshot-application-context.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:hz="http://www.hazelcast.com/schema/spring"
       xsi:schemaLocation="http://www.springframework.org/schema/beans
                           http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
                           http://www.hazelcast.com/schema/spring
                           http://www.hazelcast.com/schema/spring/hazelcast-spring-2.4.xsd">

  <hz:topic id="topic" instance-ref="webframework.cluster.slingshot" name="share_hz_test"/>
  <hz:hazelcast id="webframework.cluster.slingshot">
    <hz:config>
      <hz:group name="slingshot" password="Sh4r3_hz_Test_pwd"/>
      <hz:network port="5801" port-auto-increment="false">
        <hz:join>
          <hz:multicast enabled="false" multicast-group="224.2.2.5" multicast-port="54327"/>
          <hz:tcp-ip enabled="true">
            <hz:members>share_n1.domain,share_n2.domain</hz:members>
          </hz:tcp-ip>
        </hz:join>
        <hz:interfaces enabled="false">
          <hz:interface></hz:interface>
        </hz:interfaces>
      </hz:network>
    </hz:config>
  </hz:hazelcast>

  <bean id="webframework.cluster.clusterservice" class="org.alfresco.web.site.ClusterTopicService" init-method="init">
    <property name="hazelcastInstance" ref="webframework.cluster.slingshot" />
    <property name="hazelcastTopicName">
      <value>share_hz_test</value>
    </property>
  </bean>

</beans>
[alfresco@share_n1 ~]$

The description is basically the same as for the Multicast part. The main difference is that the multicast was disabled, the tcp-ip was enabled and there is therefore a list of members that needs to be set. This is a comma separated list of hostname or IPs that the Hazelcast will try to contact when it starts. Membership in case of Unicast is managed in the same way except that the oldest/first member will listen for new membership requests on the TCP-IP. Therefore, it’s the same principle, it’s just done differently.

Starting the first Share node in the Cluster will display the following information on the logs:

Jul 28, 2019 11:45:35 AM com.hazelcast.impl.AddressPicker
INFO: Resolving domain name 'share_n1.domain' to address(es): [127.0.0.1, 10.10.10.10]
Jul 28, 2019 11:45:35 AM com.hazelcast.impl.AddressPicker
INFO: Resolving domain name 'share_n2.domain' to address(es): [10.10.10.11]
Jul 28, 2019 11:45:35 AM com.hazelcast.impl.AddressPicker
INFO: Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [share_n1.domain/10.10.10.10, share_n2.domain/10.10.10.11, share_n1.domain/127.0.0.1]
Jul 28, 2019 11:45:35 AM com.hazelcast.impl.AddressPicker
INFO: Prefer IPv4 stack is true.
Jul 28, 2019 11:45:35 AM com.hazelcast.impl.AddressPicker
INFO: Picked Address[share_n1.domain]:5801, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5801], bind any local is true
Jul 28, 2019 11:45:36 AM com.hazelcast.system
INFO: [share_n1.domain]:5801 [slingshot] Hazelcast Community Edition 2.4 (20121017) starting at Address[share_n1.domain]:5801
Jul 28, 2019 11:45:36 AM com.hazelcast.system
INFO: [share_n1.domain]:5801 [slingshot] Copyright (C) 2008-2012 Hazelcast.com
Jul 28, 2019 11:45:36 AM com.hazelcast.impl.LifecycleServiceImpl
INFO: [share_n1.domain]:5801 [slingshot] Address[share_n1.domain]:5801 is STARTING
Jul 28, 2019 11:45:36 AM com.hazelcast.impl.TcpIpJoiner
INFO: [share_n1.domain]:5801 [slingshot] Connecting to possible member: Address[share_n2.domain]:5801
Jul 28, 2019 11:45:36 AM com.hazelcast.nio.SocketConnector
INFO: [share_n1.domain]:5801 [slingshot] Could not connect to: share_n2.domain/10.10.10.11:5801. Reason: ConnectException[Connection refused]
Jul 28, 2019 11:45:37 AM com.hazelcast.nio.SocketConnector
INFO: [share_n1.domain]:5801 [slingshot] Could not connect to: share_n2.domain/10.10.10.11:5801. Reason: ConnectException[Connection refused]
Jul 28, 2019 11:45:37 AM com.hazelcast.impl.TcpIpJoiner
INFO: [share_n1.domain]:5801 [slingshot]

Members [1] {
        Member [share_n1.domain]:5801 this
}

Jul 28, 2019 11:45:37 AM com.hazelcast.impl.LifecycleServiceImpl
INFO: [share_n1.domain]:5801 [slingshot] Address[share_n1.domain]:5801 is STARTED
2019-07-28 11:45:37,164  INFO  [web.site.ClusterTopicService] [localhost-startStop-1] Init complete for Hazelcast cluster - listening on topic: share_hz_test

Then starting a second node of the Share Cluster will display the following (still on the node1 logs):

Jul 28, 2019 11:48:31 AM com.hazelcast.nio.SocketAcceptor
INFO: [share_n1.domain]:5801 [slingshot] 5801 is accepting socket connection from /10.10.10.11:34191
Jul 28, 2019 11:48:31 AM com.hazelcast.nio.ConnectionManager
INFO: [share_n1.domain]:5801 [slingshot] 5801 accepted socket connection from /10.10.10.11:34191
Jul 28, 2019 11:48:38 AM com.hazelcast.cluster.ClusterManager
INFO: [share_n1.domain]:5801 [slingshot]

Members [2] {
        Member [share_n1.domain]:5801 this
        Member [share_n2.domain]:5801
}

Other posts of this series on Alfresco HA/Clustering:

• Basis & Architectures
• ActiveMQ
• Apache HTTPD as Load Balancer
• Share
• Solr6
• Repository

Cet article Alfresco Clustering – Share est apparu en premier sur Blog dbi services.

In previous blogs, I talked about some basis and presented some possible architectures for Alfresco, I talked about the Clustering setup for the Alfresco Repository and the Alfresco Share. In this one, I will work on the ActiveMQ layer. I recently posted something related to the setup of ActiveMQ and some initial configuration. I will therefore extend this topic in this blog with what needs to be done to have a simple Cluster for ActiveMQ. I’m not an ActiveMQ expert, I just started using it a few months ago in relation to Alfresco but still, I learned some things in this timeframe so this might be of some use.

ActiveMQ is a Messaging Server so there are therefore three sides to this component. First, there are Producers which produce messages. These messages are put in the broker’s queue which is the second side and finally there are Consumers which consume the messages from the queue. Producers and Consumers are satellites that are using the JMS broker’s queue: they are both clients. Therefore, in a standalone architecture (one broker), there is no issue because clients will always produce and consume all messages. However, if you start adding more brokers and if you aren’t doing it right, you might have producers talking to a specific broker and consumers talking to another one. To solve that, there are a few things possible:

• a first solution is to create a Network of Brokers which will allow the different brokers to forward the necessary messages between them. You can see that as an Active/Active Cluster
  • Pros: this allows ActiveMQ to support a huge architecture with potentially hundreds or thousands of brokers
  • Cons: messages are, at any point in time, only owned by one single broker so if this broker goes down, the message is lost (if there is no persistence) or will have to wait for the broker to be restarted (if there is persistence)
• the second solution that ActiveMQ supports is the Master/Slave one. In this architecture, all messages will be replicated from a Master to all Slave brokers. You can see that as something like an Active/Passive Cluster
  • Pros: messages are always processed and cannot be lost. If the Master broker is going down for any reasons, one of the Slave is instantly taking its place as the new Master with all the previous messages
  • Cons: since all messages are replicated, it’s much harder to support a huge architecture

In case of a Network of Brokers, it’s possible to use either the static or dynamic discovery of brokers:

• Static discovery: Uses the static protocol to provide a list of all URIs to be tested to discover other connections. E.g.: static:(tcp://mq_n1.domain:61616,tcp://mq_n2.domain:61616)?maxReconnectDelay=3000
• Dynamic discovery: Uses a multicast discovery agent to check for other connections. This is done using the discoveryUri parameter in the XML configuration file

I. Client’s configuration

On the client’s side, using several brokers is very simple since it’s all about using the correct broker URL. To be able to connect to several brokers, you should use the Failover Transport protocol which replaced the Reliable protocol used in ActiveMQ 3. For Alfresco, this broker URL needs to be updated in the alfresco-global.properties file. This is an example for a pretty simple URL with two brokers:

[alfresco@alf_n1 ~]$ cat $CATALINA_HOME/shared/classes/alfresco-global.properties
...
### ActiveMQ
messaging.broker.url=failover:(tcp://mq_n1.domain:61616,tcp://mq_n2.domain:61616)?timeout=3000&randomize=false&nested.daemon=false&nested.dynamicManagement=false
#messaging.username=
#messaging.password=
...
[alfresco@alf_n1 ~]$

There are a few things to note. The Failover used above is a transport layer that can be used in combination with any of the other transport methods/protocol. Here it’s used with two TCP protocol. The correct nomenclature is either:

• failover:uri1,…,uriN
  • E.g.: failover:tcp://mq_n1.domain:61616,tcp://mq_n2.domain:61616 => the simplest broker URL for two brokers with no custom options
• failover:uri1?URIOptions1,…,uriN?URIOptionsN
  • E.g.: failover:tcp://mq_n1.domain:61616?daemon=false&dynamicManagement=false&trace=false,tcp://mq_n2.domain:61616?daemon=false&dynamicManagement=true&trace=true => a more advanced broker URL with some custom options for each of the TCP protocol URIs
• failover:(uri1?URIOptions1,…,uriN?URIOptionsN)?FailoverTransportOptions
  • E.g.: failover:(tcp://mq_n1.domain:61616?daemon=false&dynamicManagement=false&trace=false,tcp://mq_n2.domain:61616?daemon=false&dynamicManagement=true&trace=true)?timeout=3000&randomize=false => the same broker URL as above but, in addition, with some Failover Transport options
• failover:(uri1,…,uriN)?FailoverTransportOptions&NestedURIOptions
  • E.g.: failover:(tcp://mq_n1.domain:61616,tcp://mq_n2.domain:61616)?timeout=3000&randomize=false&nested.daemon=false&nested.dynamicManagement=false&nested.trace=false => since ActiveMQ 5.9, it’s now possible to set the nested URIs options (here the TCP protocol options) at the end of the broker URL, they just need to be preceded by “nested.”. Nested options will apply to all URIs.

There are a lot of interesting parameters, these are some:

• Failover Transport options:
  • backup=true: initialize and keep a second connection to another broker for faster failover
  • randomize=true: will pick a new URI for the reconnect randomly from the list of URIs
  • timeout=3000: time in ms before timeout on the send operations
  • priorityBackup=true: clients will failover to other brokers in case the “primary” broker isn’t available (that’s always the case) but it will consistently try to reconnect to the “primary” one. It is possible to specify several “primary” brokers with the priorityURIs option (comma separated list)
• TCP Transport options:
  • daemon=false: specify that ActiveMQ isn’t running in a Spring or Web container
  • dynamicManagement=false: disabling the JMX management
  • trace=false: disabling the tracing

The full list of Failover Transport options is described here and the full list of TCP Transport options here.

II. Messaging Server’s configuration

I believe the simplest setup for Clustering in ActiveMQ is using the Master/Slave setup, that’s what I will talk about here. If you are looking for more information about the Network of Brokers, you can find that here. As mentioned previously, the idea behind the Master/Slave is to replicate somehow the messages to Slave brokers. To do that, there are three possible configurations:

• Shared File System: use a shared file system
• JDBC: use a Database Server
• Replicated LevelDB Store: use a ZooKeeper Server. This has been deprecated in recent versions of ActiveMQ 5 in favour of KahaDB, which is a file-based persistence Database. Therefore, this actually is linked to the first configuration above (Shared File System)

In the scope of Alfresco, you should already have a shared file system as well as a shared Database Server for the Repository Clustering… So, it’s pretty easy to fill the prerequisites for ActiveMQ since you already have them. Of course, you can use a dedicated Shared File System or dedicated Database, that’s up to your requirements.

a. JDBC

For the JDBC configuration, you will need to change the persistenceAdapter to use the dedicated jdbcPersistenceAdapter and create the associated DataSource for your Database. ActiveMQ supports some DBs like Apache Derby, DB2, HSQL, MySQL, Oracle, PostgreSQL, SQLServer or Sybase. You will also need to add the JDBC library at the right location.

[alfresco@mq_n1 ~]$ cat $ACTIVEMQ_HOME/conf/activemq.xml
<beans
  xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
  http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
  ...
  <broker xmlns="http://activemq.apache.org/schema/core" brokerName="mq_n1" dataDirectory="${activemq.data}">
    ...
    <persistenceAdapter>
      <jdbcPersistenceAdapter dataDirectory="activemq-data" dataSource="postgresql-ds"/>
    </persistenceAdapter>
    ...
  </broker>
  ...
  <bean id="postgresql-ds" class="org.postgresql.ds.PGPoolingDataSource">
    <property name="serverName" value="db_vip"/>
    <property name="databaseName" value="alfresco"/>
    <property name="portNumber" value="5432"/>
    <property name="user" value="alfresco"/>
    <property name="password" value="My+P4ssw0rd"/>
    <property name="dataSourceName" value="postgres"/>
    <property name="initialConnections" value="1"/>
    <property name="maxConnections" value="10"/>
  </bean>
  ...
</beans>
[alfresco@mq_n1 ~]$

b. Shared File System

The Shared File System configuration is, from my point of view, the simplest one to configure but for it to work properly, there are some things to note because you should use a shared file system that supports proper file lock. This means that:

• you cannot use the Oracle Cluster File System (OCFS/OCFS2) because there is no cluster-aware flock or POSIX locks
• if you are using NFS v3 or lower, you won’t have automatic failover from Master to Slave because there is no timeout and therefore the lock will never be released. You should therefore use NFS v4 instead

Additionally, you need to share the persistenceAdapter between all brokers but you cannot share the data folder completely otherwise the logs will be overwritten by all brokers (that’s bad but it’s not really an issue) and more importantly, the PID file will also be overwritten which will therefore cause issues to start/stop Slave brokers…

Therefore, configuring properly the Shared File System is all about keeping the “$ACTIVEMQ_DATA” environment variable set to the place where you want the logs and PID files to be stored (i.e. locally) and you need to overwrite the persistenceAdapter path to be on the Shared File System:

[alfresco@mq_n1 ~]$ # Root folder of the ActiveMQ binaries
[alfresco@mq_n1 ~]$ echo $ACTIVEMQ_HOME
/opt/activemq
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ # Location of the logs and PID file
[alfresco@mq_n1 ~]$ echo $ACTIVEMQ_DATA
/opt/activemq/data
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ # Location of the Shared File System
[alfresco@mq_n1 ~]$ echo $ACTIVEMQ_SHARED_DATA
/shared/file/system
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ sudo systemctl stop activemq.service
[alfresco@mq_n1 ~]$ grep -A2 "<persistenceAdapter>" $ACTIVEMQ_HOME/conf/activemq.xml
    <persistenceAdapter>
      <kahaDB directory="${activemq.data}/kahadb"/>
    </persistenceAdapter>
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ # Put the KahaDB into the Shared File System
[alfresco@mq_n1 ~]$ sed -i "s, directory=\"[^\"]*\", directory=\"${ACTIVEMQ_SHARED_DATA}/activemq/kahadb\"," $ACTIVEMQ_HOME/conf/activemq.xml
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ grep -A2 "<persistenceAdapter>" $ACTIVEMQ_HOME/conf/activemq.xml
    <persistenceAdapter>
      <kahaDB directory="/shared/file/system/activemq/kahadb"/>
    </persistenceAdapter>
[alfresco@mq_n1 ~]$
[alfresco@mq_n1 ~]$ sudo systemctl start activemq.service

Starting the Master ActiveMQ will display some information in the log of the node1 showing that it has started properly and it will listen for connections on the different transportConnector:

[alfresco@mq_n1 ~]$ cat $ACTIVEMQ_DATA/activemq.log
2019-07-28 11:34:37,598 | INFO  | Refreshing org.apache.activemq.xbean.XBeanBrokerFactory$1@9f116cc: startup date [Sun Jul 28 11:34:37 CEST 2019]; root of context hierarchy | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
2019-07-28 11:34:38,289 | INFO  | Using Persistence Adapter: KahaDBPersistenceAdapter[/shared/file/system/activemq/kahadb] | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:34:38,330 | INFO  | KahaDB is version 6 | org.apache.activemq.store.kahadb.MessageDatabase | main
2019-07-28 11:34:38,351 | INFO  | PListStore:[/opt/activemq/data/mq_n1/tmp_storage] started | org.apache.activemq.store.kahadb.plist.PListStoreImpl | main
2019-07-28 11:34:38,479 | INFO  | Apache ActiveMQ 5.15.6 (mq_n1, ID:mq_n1-36925-1564306478360-0:1) is starting | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:34:38,533 | INFO  | Listening for connections at: tcp://mq_n1:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-07-28 11:34:38,542 | INFO  | Connector openwire started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:34:38,545 | INFO  | Listening for connections at: amqp://mq_n1:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-07-28 11:34:38,546 | INFO  | Connector amqp started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:34:38,552 | INFO  | Listening for connections at: stomp://mq_n1:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-07-28 11:34:38,553 | INFO  | Connector stomp started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:34:38,556 | INFO  | Listening for connections at: mqtt://mq_n1:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-07-28 11:34:38,561 | INFO  | Connector mqtt started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:34:38,650 | WARN  | ServletContext@o.e.j.s.ServletContextHandler@11841b15{/,null,STARTING} has uncovered http methods for path: / | org.eclipse.jetty.security.SecurityHandler | main
2019-07-28 11:34:38,710 | INFO  | Listening for connections at ws://mq_n1:61614?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.ws.WSTransportServer | main
2019-07-28 11:34:38,712 | INFO  | Connector ws started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:34:38,712 | INFO  | Apache ActiveMQ 5.15.6 (mq_n1, ID:mq_n1-36925-1564306478360-0:1) started | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:34:38,714 | INFO  | For help or more information please see: http://activemq.apache.org | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:34:39,118 | INFO  | No Spring WebApplicationInitializer types detected on classpath | /admin | main
2019-07-28 11:34:39,373 | INFO  | ActiveMQ WebConsole available at http://0.0.0.0:8161/ | org.apache.activemq.web.WebConsoleStarter | main
2019-07-28 11:34:39,373 | INFO  | ActiveMQ Jolokia REST API available at http://0.0.0.0:8161/api/jolokia/ | org.apache.activemq.web.WebConsoleStarter | main
2019-07-28 11:34:39,402 | INFO  | Initializing Spring FrameworkServlet 'dispatcher' | /admin | main
2019-07-28 11:34:39,532 | INFO  | No Spring WebApplicationInitializer types detected on classpath | /api | main
2019-07-28 11:34:39,563 | INFO  | jolokia-agent: Using policy access restrictor classpath:/jolokia-access.xml | /api | main
[alfresco@mq_n1 ~]$

Then starting a Slave will only display the information on the node2 logs that there is already a Master running and therefore the Slave is just waiting and it’s not listening for now:

[alfresco@mq_n2 ~]$ cat $ACTIVEMQ_DATA/activemq.log
2019-07-28 11:35:53,258 | INFO  | Refreshing org.apache.activemq.xbean.XBeanBrokerFactory$1@9f116cc: startup date [Sun Jul 28 11:35:53 CEST 2019]; root of context hierarchy | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
2019-07-28 11:35:53,986 | INFO  | Using Persistence Adapter: KahaDBPersistenceAdapter[/shared/file/system/activemq/kahadb] | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:35:53,999 | INFO  | Database /shared/file/system/activemq/kahadb/lock is locked by another server. This broker is now in slave mode waiting a lock to be acquired | org.apache.activemq.store.SharedFileLocker | main
[alfresco@mq_n2 ~]$

Finally stopping the Master will automatically transform the Slave into a new Master, without any human interaction. From the node2 logs:

[alfresco@mq_n2 ~]$ cat $ACTIVEMQ_DATA/activemq.log
2019-07-28 11:35:53,258 | INFO  | Refreshing org.apache.activemq.xbean.XBeanBrokerFactory$1@9f116cc: startup date [Sun Jul 28 11:35:53 CEST 2019]; root of context hierarchy | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
2019-07-28 11:35:53,986 | INFO  | Using Persistence Adapter: KahaDBPersistenceAdapter[/shared/file/system/activemq/kahadb] | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:35:53,999 | INFO  | Database /shared/file/system/activemq/kahadb/lock is locked by another server. This broker is now in slave mode waiting a lock to be acquired | org.apache.activemq.store.SharedFileLocker | main
  # The ActiveMQ Master on node1 has been stopped here (11:37:10)
2019-07-28 11:37:11,166 | INFO  | KahaDB is version 6 | org.apache.activemq.store.kahadb.MessageDatabase | main
2019-07-28 11:37:11,187 | INFO  | PListStore:[/opt/activemq/data/mq_n2/tmp_storage] started | org.apache.activemq.store.kahadb.plist.PListStoreImpl | main
2019-07-28 11:37:11,316 | INFO  | Apache ActiveMQ 5.15.6 (mq_n2, ID:mq_n2-41827-1564306631196-0:1) is starting | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:37:11,370 | INFO  | Listening for connections at: tcp://mq_n2:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-07-28 11:37:11,372 | INFO  | Connector openwire started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:37:11,379 | INFO  | Listening for connections at: amqp://mq_n2:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-07-28 11:37:11,381 | INFO  | Connector amqp started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:37:11,386 | INFO  | Listening for connections at: stomp://mq_n2:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-07-28 11:37:11,387 | INFO  | Connector stomp started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:37:11,390 | INFO  | Listening for connections at: mqtt://mq_n2:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-07-28 11:37:11,391 | INFO  | Connector mqtt started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:37:11,485 | WARN  | ServletContext@o.e.j.s.ServletContextHandler@2cfbeac4{/,null,STARTING} has uncovered http methods for path: / | org.eclipse.jetty.security.SecurityHandler | main
2019-07-28 11:37:11,547 | INFO  | Listening for connections at ws://mq_n2:61614?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.ws.WSTransportServer | main
2019-07-28 11:37:11,548 | INFO  | Connector ws started | org.apache.activemq.broker.TransportConnector | main
2019-07-28 11:37:11,556 | INFO  | Apache ActiveMQ 5.15.6 (mq_n2, ID:mq_n2-41827-1564306631196-0:1) started | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:37:11,558 | INFO  | For help or more information please see: http://activemq.apache.org | org.apache.activemq.broker.BrokerService | main
2019-07-28 11:37:11,045 | INFO  | No Spring WebApplicationInitializer types detected on classpath | /admin | main
2019-07-28 11:37:11,448 | INFO  | ActiveMQ WebConsole available at http://0.0.0.0:8161/ | org.apache.activemq.web.WebConsoleStarter | main
2019-07-28 11:37:11,448 | INFO  | ActiveMQ Jolokia REST API available at http://0.0.0.0:8161/api/jolokia/ | org.apache.activemq.web.WebConsoleStarter | main
2019-07-28 11:37:11,478 | INFO  | Initializing Spring FrameworkServlet 'dispatcher' | /admin | main
2019-07-28 11:37:11,627 | INFO  | No Spring WebApplicationInitializer types detected on classpath | /api | main
2019-07-28 11:37:11,664 | INFO  | jolokia-agent: Using policy access restrictor classpath:/jolokia-access.xml | /api | main
[alfresco@mq_n2 ~]$

You can of course customize ActiveMQ as per your requirements, remove some connectors, setup SSL, aso… But that’s not really the purpose of this blog.

Other posts of this series on Alfresco HA/Clustering:

• Basis & Architectures
• ActiveMQ
• Apache HTTPD as Load Balancer
• Share
• Solr6
• Repository

Cet article Alfresco Clustering – ActiveMQ est apparu en premier sur Blog dbi services.