Pour ajouter un core, voir SolrMaintenance.

Installing Solr 6.x from upstream on Debian Jessie

Upstream mirror: http://www-us.apache.org/dist/lucene/solr/

  1. Download files:

    SOLR_VERSION=6.5.1
    wget http://www-us.apache.org/dist/lucene/solr/$SOLR_VERSION/solr-$SOLR_VERSION.tgz -O solr.tgz
    wget http://www-us.apache.org/dist/lucene/solr/$SOLR_VERSION/solr-$SOLR_VERSION.tgz.asc -O solr.tgz.asc
  2. Verify that the download files:

    wget http://www-us.apache.org/dist/lucene/solr/$SOLR_VERSION/KEYS
    gpg --import KEYS
    gpg --verify solr.tgz.asc solr.tgz
  3. Install dependencies and needed programs:

    apt install ca-certificates-java=20161107~bpo8+1 # For some reason, the 2014 version was marked as held or prioritized
    apt install openjdk-8-jre-headless openjdk-8-jdk-headless tomcat8
    update-alternatives --config java # choose openjdk-8 as the default
  4. Install solr

    SOLR_VERSION=6.5.1
    tar xzf solr.tgz
    cp -r solr-$SOLR_VERSION /opt/solr-$SOLR_VERSION
    # The script fails without this, as upstreams packs the folder as solr-X.Y.Z instead of solr alone
    ln --symbolic /opt/solr-$SOLR_VERSION /opt/solr
    ./solr-$SOLR_VERSION/bin/install_solr_service.sh solr.tgz -f -n
    # The script doesn't work as well as advertised, let's cludge it back in shape
    rm /opt/solr
    ln --symbolic /opt/solr-$SOLR_VERSION /opt/solr
    service solr start #listens on 8983 by default
    # Service status
    service solr status
    # Different info from init
    /etc/init.d/solr status
    # The admin panel should be accessible at http://solr0.koumbit.net:8983/solr/#/
  5. Hardening

    # https://cwiki.apache.org/confluence/display/solr/Taking+Solr+to+Production
    # SSL
    # https://cwiki.apache.org/confluence/display/solr/Enabling+SSL
    apt -t jessie-backports install certbot
    SOLR_DOMAIN=solr0.koumbit.net
    certbot --standalone certonly -d $SOLR_DOMAIN
    # Register e-mail: ssl@rt.koumbit.net, see SslService/LetsEncrypt
    # Convert to to pkcs and jks. @TODO: Post-certbot renew hook. You will prompted for a secret when exporting to pkcs12
    # Note: It's easiest to  keep SECRETs the same
    openssl pkcs12 -export -in /etc/letsencrypt/live/$SOLR_DOMAIN/fullchain.pem -inkey /etc/letsencrypt/live/$SOLR_DOMAIN/privkey.pem -out /var/solr/pkcs.p12 -name $SOLR_DOMAIN
    keytool -importkeystore -deststorepass SECRET -destkeypass SECRET -destkeystore /var/solr/solr.keystore -srckeystore /var/solr/pkcs.p12 -srcstoretype PKCS12 -srcstorepass STORE_PASS -alias $SOLR_DOMAIN
    # Update solr configuration
    vim /etc/default/solr.in.sh
    # Set the following lines:
    SOLR_SSL_KEY_STORE=/var/solr/solr.keystore
    SOLR_SSL_KEY_STORE_PASSWORD=SECRET
    SOLR_SSL_KEY_STORE_TYPE=JKS
    SOLR_SSL_TRUST_STORE=/var/solr/solr.keystore
    SOLR_SSL_TRUST_STORE_PASSWORD=SECRET
    SOLR_SSL_TRUST_STORE_TYPE=JKS
    # Leave SOLR_SSL_NEED_CLIENT_AUTH & SOLR_SSL_WANT_CLIENT_AUTH set to false / commented out\
    /etc/init.d/solr restart
    # Admin page is now accessible on https://solr0.koumbit.net only
    # Create the file /var/solr/data/security.conf
    ###
    {
    "authentication":{
       "blockUnknown": true,
       "class":"solr.BasicAuthPlugin",
       "credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
    },
    "authorization":{
       "class":"solr.RuleBasedAuthorizationPlugin",
       "permissions":[{"name":"security-edit",
          "role":"admin"}],
       "user-role":{"solr":"admin"}
    }}
    ###
    # Default pass: SolrRocks
    # Not sure how to generate others, mkpasswd -m sha-256 -R 20 PASSWORD SALT16 | base64 doesn't seem to do the trick
    # Passwords can be set through the admin interface anyway. Or not?
    service solr restart
  6. Renouvellement automatique des certificat LetsEncrypt avec cerbot

    mkdir -p /etc/letsencrypt/renewal-hooks/post
    touch /etc/letsencrypt/renewal-hooks/solr.sh
    chmod +x /etc/letsencrypt/renewal-hooks/solr.sh
    1. Ajouter ce contenu et modifier selon vos paramètres:

      set -e
      
      for domain in $RENEWED_DOMAINS; do
              case $domain in
              xxx.koumbit.net)
                      SOLR_DOMAIN=xxx.koumbit.net
                      SECRET=xxx
                      openssl pkcs12 -export -in /etc/letsencrypt/live/$SOLR_DOMAIN/fullchain.pem -inkey /etc/letsencrypt/live/$SOLR_DOMAIN/privkey.pem -out /var/solr/pkcs.p12 -name $SOLR_DOMAIN -password pass:$SECRET
                      rm /var/solr/solr.keystore
                      keytool -importkeystore -deststorepass $SECRET -destkeypass $SECRET -destkeystore /var/solr/solr.keystore -srckeystore /var/solr/pkcs.p12 -srcstoretype PKCS12 -srcstorepass $SECRET -alias $SOLR_DOMAIN
                      service solr restart
                      ;;
              esac
      done

Updating solr version from upstream

Note: avant de faire le procédure, checker si les mises à jour ont des impications (eg. besoin de re-indexer, etc.).

export OLD_SOLR_VERSION=6.5.1
export NEW_SOLR_VERSION=6.6.2
cd ~/ && wget http://www-us.apache.org/dist/lucene/solr/$NEW_SOLR_VERSION/solr-$NEW_SOLR_VERSION.tgz
wget http://www-us.apache.org/dist/lucene/solr/$NEW_SOLR_VERSION/solr-$NEW_SOLR_VERSION.tgz.asc
wget http://www-us.apache.org/dist/lucene/solr/$NEW_SOLR_VERSION/KEYS
gpg --import KEYS
gpg --verify solr-$NEW_SOLR_VERSION.tgz.asc solr-$NEW_SOLR_VERSION.tgz
tar xzf solr-$NEW_SOLR_VERSION.tgz
service solr stop
./solr-$NEW_SOLR_VERSION/bin/install_solr_service.sh solr-$NEW_SOLR_VERSION.tgz -f -n
service solr start

Installing Solr 3.6 from package on Debian Squeeze with Tomcat 6

  1. Installation des packages Debian de Solr, Tomcat et leurs dépendances:

    echo "deb http://ftp.at.debian.org/debian wheezy main contrib non-free" >> /etc/apt/sources.list
    apt-get update
    apt-get -t wheezy install solr-tomcat
  2. Puisque le package Debian solr-common ne contient pas Tika (This package [...] omiting dataimporthandler-extras, clustering, extraction and velocity due to missing dependencies), il faut aller chercher le tarball complet de Solr.

    cd /usr/share
    wget http://apache.mirror.iweb.ca/lucene/solr/3.6.1/apache-solr-3.6.1.tgz
    tar zxvf apache-solr-3.6.1.tgz && rm apache-solr-3.6.1.tgz
    ln -s apache-solr-3.6.1 apache-solr
  3. Ajouter un rôle et un utilisateur pour l'administration web ainsi que Munin pour la création de graphes. Le fichier '/etc/tomcat6/tomcat-users.xml' sera:

    <?xml version='1.0' encoding='utf-8'?>
    <tomcat-users>
    
      <role rolename="admin"/>
      <user username="admin" password="xxxxx" roles="admin"/>
    
      <role rolename="manager"/>
      <user username="munin" password="xxxxx" roles="manager"/>
    
    </tomcat-users>
  4. Sécuriser l'accès à Solr en ajoutant ces lignes à la fin du fichier '/etc/solr/web.xml', avant la fermeture du tag </web-app>:

      <login-config>
        <auth-method>BASIC</auth-method>
        <realm-name>Solr Authentication</realm-name>
      </login-config>
    
      <security-constraint>
        <web-resource-collection>
          <web-resource-name>Solr Admin</web-resource-name>
          <url-pattern>/*</url-pattern>
        </web-resource-collection>
          <auth-constraint>
          <role-name>admin</role-name>
        </auth-constraint>
      </security-constraint>
    
    </web-app>
  5. Éditer le fichier '/etc/solr/solr.xml' pour qu'il soit comme ceci, puisqu'il n'y a pas encore de core à définir:

    <?xml version="1.0" encoding="UTF-8" ?>
    <solr persistent="false" sharedLib="lib">
      <cores adminPath="/admin/cores">
      </cores>
    </solr>
  6. Créer un répertoire pour recevoir la configuration des cores:

    mkdir /etc/solr/cores
  7. Éditer le fichier '/etc/solr/conf/stopwords.txt' pour qu'il soit comme StopWords:

    cp -p /etc/solr/conf/stopwords.txt /etc/solr/conf/stopwords.txt.orig
    vi /etc/solr/conf/stopwords.txt
  8. Redémarrer Tomcat:

    invoke-rc.d tomcat6 restart

Références:

Installing Solr 1.4.1 from sources on Debian Squeeze with Tomcat 6

  1. Installation de Tomcat:

    apt-get install tomcat6
  2. Installation de Solr:

    cd /usr/share
    wget http://archive.apache.org/dist/lucene/solr/1.4.1/apache-solr-1.4.1.tgz
    tar zxvf apache-solr-1.4.1.tgz
    rm       apache-solr-1.4.1.tgz
    ln -s    apache-solr-1.4.1 solr
    mkdir /etc/solr
    mkdir /etc/solr/cores
    mkdir /var/lib/solr
    cp -R /usr/share/solr/example/solr/conf /etc/solr/
  3. Éditer le fichier '/etc/solr/conf/stopwords.txt' pour qu'il soit comme StopWords:

    cp -p /etc/solr/conf/stopwords.txt /etc/solr/conf/stopwords.txt.orig
    vi /etc/solr/conf/stopwords.txt
  4. Créer le fichier '/etc/solr/solr.xml' tel que voici:

    <?xml version="1.0" encoding="UTF-8" ?>
    <solr persistent="false" sharedLib="lib">
      <cores adminPath="/admin/cores">
      </cores>
    </solr>
  5. Créer un symlink pour le fichier 'solr.xml':

    cd /usr/share/solr
    ln -s /etc/solr/solr.xml
  6. Créer un symlink pour le fichier 'solr.war':

    cd /var/lib/tomcat6/webapps
    ln -s /usr/share/solr/dist/apache-solr-1.4.1.war solr.war
  7. Créer le fichier '/etc/solr/solr-tomcat.xml' tel que voici:

    <Context docBase="/var/lib/tomcat6/webapps/solr.war" debug="0" privileged="true" allowLinking="true" crossContext="true" >
    
      <!-- make symlinks work in Tomcat -->
      <Resources className="org.apache.naming.resources.FileDirContext" allowLinking="true" />
    
      <Environment name="solr/home" type="java.lang.String" value="/usr/share/solr" override="true" />
    
    </Context>
  8. Créer un symlink pour le fichier 'solr-tomcat.xml':

    cd /etc/tomcat6/Catalina/localhost/
    ln -s /etc/solr/solr-tomcat.xml solr.xml
  9. Créer le fichier '/etc/solr/tomcat.policy' tel que voici:

    grant codeBase "file:/usr/share/solr/-" {
      permission java.lang.RuntimePermission "modifyThread";
      permission java.lang.RuntimePermission "accessClassInPackage.org.apache.tomcat.util.http";
      permission java.util.PropertyPermission "sun.arch.data.model", "read";
      permission java.util.PropertyPermission "java.io.tmpdir", "read";
      permission java.util.PropertyPermission "user.dir", "read";
      permission java.util.PropertyPermission "solr.*", "read";
      permission java.util.PropertyPermission "org.apache.lucene.lockDir", "read,write";
      permission java.util.PropertyPermission "org.apache.lucene.store.FSDirectoryLockFactoryClass", "read";
      permission java.io.FilePermission "/usr/share/java", "read";
      permission java.io.FilePermission "/usr/share/java/-", "read";
      permission java.io.FilePermission "/usr/share/maven-repo/-", "read";
      permission java.io.FilePermission "/var/log/tomcat6/-", "read,write";
      permission java.io.FilePermission "/var/lib/tomcat6/webapps/solr/-", "read";
      permission java.io.FilePermission "/var/lib/tomcat6/temp/-", "read,write";
      permission java.io.FilePermission "/etc/solr/-", "read";
      permission java.io.FilePermission "/usr/share/solr/-", "read";
      permission java.io.FilePermission "/usr/share/solr", "read";
      permission java.io.FilePermission "/var/lib/solr", "read,write,delete";
      permission java.io.FilePermission "/var/lib/solr/-", "read,write,delete";
      permission javax.management.MBeanServerPermission "findMBeanServer";
      permission javax.management.MBeanPermission "org.apache.solr.core.*", "*";
      permission javax.management.MBeanTrustPermission "register";
    
      // dataimporthandler
      permission java.io.FilePermission "/usr/share/solr/conf/dataimport.properties", "read,write,delete";
      // really ugly, but we would need to patch solr to get around this:
      permission java.io.FilePermission "/etc/solr/conf/dataimport.properties", "read,write,delete";
      // needed to access mysql via dataimporthandler
      permission java.net.SocketPermission "127.0.0.1:3306", "connect,resolve" ;
    
      /* needed by admin/get-properties.jsp 
      permission java.util.PropertyPermission "*", "read,write"; */
    
      /* for admin/threaddump.jsp
      permission java.lang.management.ManagementPermission "monitor"; */
    };
  10. Créer un symlink pour le fichier 'tomcat.policy':

    cd /etc/tomcat6/policy.d/
    ln -s /etc/solr/tomcat.policy 05solr.policy
  11. Ajouter un rôle et un utilisateur pour l'administration web ainsi que Munin pour la création de graphes. Le fichier '/etc/tomcat6/tomcat-users.xml' sera:

    <?xml version='1.0' encoding='utf-8'?>
    <tomcat-users>
    
      <role rolename="admin"/>
      <user username="admin" password="xxxxx" roles="admin"/>
    
      <role rolename="manager"/>
      <user username="munin" password="xxxxx" roles="manager"/>
    
    </tomcat-users>
  12. Déplacer le fichier 'web.xml' et lui créer un symlink:

    mv /var/lib/tomcat6/webapps/solr/WEB-INF/web.xml /etc/solr/web.xml
    cd /var/lib/tomcat6/webapps/solr/WEB-INF/
    ln -s /etc/solr/web.xml
  13. Sécuriser l'accès à Solr en ajoutant ces lignes à la fin du fichier '/etc/solr/web.xml', avant la fermeture du tag </web-app>:

      <login-config>
        <auth-method>BASIC</auth-method>
        <realm-name>Solr Authentication</realm-name>
      </login-config>
    
      <security-constraint>
        <web-resource-collection>
          <web-resource-name>Solr Admin</web-resource-name>
          <url-pattern>/*</url-pattern>
        </web-resource-collection>
          <auth-constraint>
          <role-name>admin</role-name>
        </auth-constraint>
      </security-constraint>
    
      <!-- Add below a security-constraint for each core. -->
    
    </web-app>
  14. Redémarrer Tomcat:

    invoke-rc.d tomcat6 restart

Références:

Installing Tika on the web server

Instead of extracting text on the solr server, it's possible to extract it on the web server and pass it to the solr server for indexing.

Advantages: avoids sending potentially large files to a remote solr server (eg. large video files with only a small amount of text metadata)

Disadvantages: web server needs to execute java programs

Note: the module search_api_attachments is able to index attachments using either tika or the text extraction library built into apache solr since version 1.3, but the module apachesolr_attachments still requires tika.


CategoryConfiguration

SolrConfiguration (last edited 2018-08-15 13:52:59 by kienan)