ServerStack » Linux https://www.serverstack.com/blog Scalability Blog Sun, 03 Mar 2013 23:58:43 +0000 en-US hourly 1 http://wordpress.org/?v=4.2.2 Automatic WordPress Updates Using FTP/FTPS or SSH https://www.serverstack.com/blog/2013/02/11/automatic-wordpress-updates-using-ftpftps-or-ssh/ https://www.serverstack.com/blog/2013/02/11/automatic-wordpress-updates-using-ftpftps-or-ssh/#comments Mon, 11 Feb 2013 06:43:37 +0000 https://www.serverstack.com/blog/?p=382 Introduction When working with WordPress in a more secure environment where websites are not entirely world-writable, you will notice upgrades request FTP or FTPS credentials as the server itself does not typically have write access in properly-configured environments. Entering these credentials for every upgrade can become quite tedious, and WordPress has implemented some constants you can define within wp-config.php to make upgrades automatic. It should be noted here that you can also make upgrades ... ]]>

Introduction

When working with WordPress in a more secure environment where websites are not entirely world-writable, you will notice upgrades request FTP or FTPS credentials as the server itself does not typically have write access in properly-configured environments. Entering these credentials for every upgrade can become quite tedious, and WordPress has implemented some constants you can define within wp-config.php to make upgrades automatic.

It should be noted here that you can also make upgrades automatic by setting the file ownership of all files within the WordPress directory to the same user/group under which the webserver is running. THIS IS HORRIBLE SECURITY PRACTICE!

While storing your FTP credentials for a specific user can also be considered insecure in certain instances, it can be a very safe method to automate WordPress updates under the proper conditions. Some general considerations which can make stored credentials MUCH more secure include:

FTP:

1. Creating a separate user and restricting its access to only allow connections from localhost
2. Ensuring your FTP daemon is “chrooting” the user to their own directory only
3. Configuring your FTP daemon to listen only on localhost, thus preventing external connections
4. Using something more secure than FTP, such as SSH — Yes, we realize this one does not actually improve FTP security :)

SSH:

1. Creating a separate user (usually an alias with the same UID, different GID) and restricting access to only localhost for this specific user in sshd_config with the AllowHosts option
2. Creating some advanced SSH configuration such as chrooted SFTP-only users
3. Using public key authentication, which can be further secured by specifying a “from” address in the user’s authorized_keys file

There are several other ways one can make their FTP/FTPS or SSH setup more secure, but they are far beyond the scope of this post and can vary greatly in their application due to the hosting environment and several other factors. We are going to assume you’re already working with a secure setup for the purposes of this guide.

WordPress Upgrade Constants

From the WordPress Codex, the following constants are available to define FTP and SSH credentials in wp-config.php:

FS_METHOD This setting forces the filesystem (or connection) method, and you probably won’t need to adjust or define it. It can be one of: “direct”, “ssh2″, “ftpext”, or “ftpsockets”. WordPress will automatically determine the proper method using the following preferential order:
(Primary Preference) “direct” causes the use of direct file I/O requests from within PHP, but this requires the webserver to have write access to your WordPress installation, which is NOT recommended. This setting will be chosen automatically when the permissions allow.
(Secondary Preference) “ssh2″ allows forcing usage of the SSH2 PHP extension if installed (via PECL).
(3rd Preference) “ftpext” allows forcing the usage of the FTP PHP extension (this is usually the default when you connect via FTP/FTPS).
(4th Preference) “ftpsockets” utilizes the PHP sockets class for FTP access (far less common, but can resolve FTP connection issues in rare cases).

FTP_BASE is the full path to the “base” (absolute path) folder of your WordPress installation.

FTP_CONTENT_DIR is the full path to the wp-content folder of your WordPress installation.

FTP_PLUGIN_DIR is the full path to the plugins folder of your WordPress installation.

FTP_PUBKEY is the full path to your SSH public key.

FTP_PRIKEY is the full path to your SSH private key.

FTP_USER is either your FTP or SSH username, depending on which method you use.

FTP_PASS is the password for the username entered for FTP_USER. If you are using SSH public key authentication, this can be left blank.

FTP_HOST is the hostname[:port] combination for your SSH/FTP server. The default FTP port is 21 and the default SSH port is 22. You only need to specify the port if using a non-standard one.

FTP_SSL is only for FTPS connections, and should not be defined unless you have already configured your FTP daemon to support TLS. Note – SFTP is NOT the same thing, so make sure you do not confuse the two.

Here’s an example of the most common configuration options with sample values so you can see the proper method of defining them within wp-config.php:

define('FS_METHOD', 'ftpext');
define('FTP_BASE', '/path/to/wordpress/');
define('FTP_CONTENT_DIR', '/path/to/wordpress/wp-content/');
define('FTP_PLUGIN_DIR ', '/path/to/wordpress/wp-content/plugins/');
define('FTP_PUBKEY', '/home/username/.ssh/id_rsa.pub');
define('FTP_PRIKEY', '/home/username/.ssh/id_rsa');
define('FTP_USER', 'username');
define('FTP_PASS', 'password');
define('FTP_HOST', 'ftp.example.org');
define('FTP_SSL', false);

To configure FTP/FTPS, you simply define the necessary constants from the list above in wp-config.php. A minimal configuration requires at least FTP_BASE, FTP_USER, FTP_PASS and FTP_HOST (usually 127.0.0.1). Enter these required constants, also adding FTP_SSL (true) if using FTPS, then your next upgrades should be automatic, and you should no longer be prompted to enter these details.

Enabling SSH support in WordPress Using the PECL SSH2 extension

Most users are not aware of this, but WordPress already supports SSH connections in addition to FTP/FTPS by simply enabling the SSH2 extension in PHP. Let’s begin by installing the SSH2 extension via PECL.

On RHEL/CentOS, you will need the php-devel, php-pear and libssh2/libssh2-devel packages and a working compiler/development libraries if you installed PHP via Yum (RPM-based installation):

# yum install php-devel php-pear gcc gcc-c++ make automake autoconf pcre-devel re2c libssh2 libssh2-devel

With the necessary prerequisites installed, you can now use the CLI tool ‘pecl’ to automagically install the extension for you:

# pecl install ssh2-0.12

The reason we need to define the version here is to avoid an error message about the extension being in “beta,” since there was never a release of this particular extension that was labeled as “stable.” Once the installation completes successfully, you’ll be presented with a success message that instructs you to enable the extension in php.ini. When using CentOS, each extension’s INI file is stored separately from the main php.ini for cleanliness and easy addition/removal of extensions. To update /etc/php.d/ssh2.ini, we will use the following command:

# echo "extension=ssh2.so" > /etc/php.d/ssh2.ini

Now, running ‘php -m’ should show the SSH2 extension in the list of extensions. If you see it there, you must now restart your PHP processor (we’ll assume it’s Apache):

# /etc/init.d/httpd restart

You now have the SSH2 extension installed and enabled. If you have not already entered any constants in wp-config.php, you can attempt an upgrade or plugin installation/deletion and you will now see a new radio button that says SSH, in addition to the FTP and FTPS choices you’ve always had. To complete this configuration, you can now just enter the same minimal options used above, possibly including the FS_METHOD constant (ssh2) to ensure only SSH connections are attempted. However, we assume you would rather use the most secure method you can, so let’s configure SSH with public key authentication.

We’ll start by generating a public/private keypair, which we will later define in wp-config.php:

# ssh-keygen -t rsa -b 4096

Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): /home/user1/wp_rsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/user1/wp_rsa.
Your public key has been saved in /home/user1/wp_rsa.pub.
The key fingerprint is:
xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx root@server1.example.com

The location of the keys should be somewhere outside of your webroot, so the user’s home directory is usually a safe choice. You should NOT enter a password here, as there have been many issues getting passworded SSH keys to work properly with WordPress. After creating the keypair, we need to make it readable by the webserver (we’ll assume your webserver runs under the “apache” user for simplicity):

# chown user1:apache /home/user1/wp_rsa
# chown user1:apache /home/user1/wp_rsa.pub
# chmod 0640 /home/user1/wp_rsa
# chmod 0640 /home/user1/wp_rsa.pub

Next, you just need to edit wp_rsa.pub to specify the ‘from=’ option and add the contents to the authorized_keys file in /home/user1/.ssh/authorized_keys:

# vim /home/user1/wp_rsa.pub

You can use whichever editor you please (vi, nano, emacs, etc), so there’s no need to cry. Once you’ve opened the file, add the following ‘from=’ restriction at the beginning of the line (there should only be one very long line) right before ssh-rsa and the key data:

from="127.0.0.1" ssh-rsa ...

Now, we can actually place the public key’s contents in the user’s authorized_keys file:

# mkdir /home/user1/.ssh
# chown user1:user1 /home/user1/.ssh/
# chmod 0700 /home/user1/.ssh/
# cat /home/user1/wp_rsa.pub >> /home/user1/.ssh/authorized_keys
# chown user1:user1 /home/user1/.ssh/authorized_keys
# chmod 0644 /home/user1/.ssh/authorized_keys

As long as PubkeyAuthentication is enabled in sshd_config (default), you should now be ready to configure wp-config.php for automatic SSH upgrades:

define('FTP_PUBKEY','/home/user1/wp_rsa.pub');
define('FTP_PRIKEY','/home/user1/wp_rsa');
define('FTP_USER','user1');
define('FTP_PASS','');
define('FTP_HOST','127.0.0.1:22');

From now on, installing/removing/upgrading WordPress and its plugins should no longer prompt you for credentials. Happy blogging!

]]>
https://www.serverstack.com/blog/2013/02/11/automatic-wordpress-updates-using-ftpftps-or-ssh/feed/ 0
Debugging NFS File Access on Server and Client Side https://www.serverstack.com/blog/2012/11/21/debugging-nfs-file-access-on-server-and-client-side/ https://www.serverstack.com/blog/2012/11/21/debugging-nfs-file-access-on-server-and-client-side/#comments Wed, 21 Nov 2012 15:46:59 +0000 https://www.serverstack.com/blog/?p=244 In an environment where a lot of servers share same NFS mount, keeping track of which server created or deleted a file is important.  By default, NFS client configuration does not have an option for access logging in Linux distributions.  To mitigate this situation, you can use a utility called rpcdebug. To see a list of all supported modules and flags: # rpcdebug -vusage: rpcdebug [-v] [-h] [-m module] [-s flags...|-c flags...] set or cancel ... ]]>

In an environment where a lot of servers share same NFS mount, keeping track of which server created or deleted a file is important.  By default, NFS client configuration does not have an option for access logging in Linux distributions.  To mitigate this situation, you can use a utility called rpcdebug.
To see a list of all supported modules and flags:

# rpcdebug -vusage: rpcdebug [-v] [-h] [-m module] [-s flags...|-c flags...]
set or cancel debug flags.
Module     Valid flags

rpc        xprt call debug nfs auth bind sched trans svcsock svcdsp misc cache all

nfs        vfs dircache lookupcache pagecache proc xdr file root callback client mount fscache pnfs pnfs_ld all

nfsd       sock fh export svc proc fileop auth repcache xdr lockd all

nlm        svc client clntlock svclock monitor clntsubs svcsubs hostcache xdr all

In our case we are interested in modules nfsd for the server side, and nfs for client side.

The vfs flag would be more descriptive than proc flag, and using all could potentially overflow your logs and slow down the system on a busy NFS server or client.

To enable access logging on server side:

[root@nfs-server nfs]# rpcdebug -m nfsd -s proc

You can now test logging on server by creating and deleting a file from an NFS client:

[root@nfs-client ~]# cd /mnt/nfs && touch test && rm -rf test

The debug messages will be logged on the NFS server to /var/log/messages by default.  Now that you have enabled the debug logging, you can find when a particular file was deleted, since information on when the file was created could be obtained via stat command, but recovering information on a deleted file could be impossible after a while, especially if there is no way of knowing whether a file was deleted in the first place.

For our particular example, the filename is test :

[root@nfs-server nfs]# grep test /var/log/messages
Nov 17 16:26:24 nfs-server kernel: [8908851.761009] NFSD: nfsd4_open filename test op_stateowner           (null)

Nov 17 16:26:24 nfs-server kernel: [8908851.793629] NFSD: nfsd4_open_confirm on file test

Nov 17 16:26:24 nfs-server kernel: [8908851.821583] NFSD: nfsd4_close on file test

Access logging on NFS client side requires use of nfs module :

[root@nfs-client ~]# rpcdebug -m nfs -s proc
[root@nfs-client ~]# cd /mnt/nfs && touch test && rm -rf test
[root@nfs-client nfs]# grep test /var/log/messages
Nov 17 21:34:51 nfs-client kernel: NFS call  lookup test

Nov 17 21:34:51 nfs-client kernel: NFS call  create test

Nov 17 21:34:53 nfs-client kernel: NFS call  remove test

 

Using vfs flag, you would be searching for instances of unlink or safe_remove in log files.

If you would like to keep your debug messages in a separate file, add the following line to /etc/syslog.conf ( /etc/rsyslog.conf if you are running rsyslog on Fedora / openSUSE / Debian / Ubuntu  / *BSD ):

*.debug                                 /var/log/debug.log

To disable all debugging and stop logging:

On server side:

rpcdebug -m nfsd -c

On client side:

rpcdebug -m nfs -c

As simple as these tools are, they are an important part of auditing in an enterprise environment, where multiple applications could be accessing same network storage mount from different servers.

]]>
https://www.serverstack.com/blog/2012/11/21/debugging-nfs-file-access-on-server-and-client-side/feed/ 0
The Joy of ex https://www.serverstack.com/blog/2012/10/08/the-joy-of-ex/ https://www.serverstack.com/blog/2012/10/08/the-joy-of-ex/#comments Mon, 08 Oct 2012 20:34:34 +0000 https://www.serverstack.com/blog/?p=193 Fellow nerds!  If there’s anything that has a tendency to work up linux guys, it’s text editors.  And in general, people tend to traditionally gravitate towards two camps: camp emacs or camp vi.   While other newer software exists, such as nano, pico, or joe – it’s important to be comfortable using vi and emacs.  vi will be available pretty much everywhere you need it, and emacs, when installed, is intensely featureful.   Besides, ... ]]>

Fellow nerds!  If there’s anything that has a tendency to work up linux guys, it’s text editors.  And in general, people tend to traditionally gravitate towards two camps: camp emacs or camp vi.   While other newer software exists, such as nano, pico, or joe – it’s important to be comfortable using vi and emacs.  vi will be available pretty much everywhere you need it, and emacs, when installed, is intensely featureful.   Besides, using these two editors are a bit of right of passage — you don’t want to look like a nano-using weakling the rest of your life, do you?  (ducks)

Your humble narrator is a vim user, and so we’ll take a look at a few relics from the past that are still used day to day.

Bill Joy (co-founder of Sun) wrote vi over 30 years ago, and us system administrators have been self-flagellating with it ever since.  Prior to vi, people at that time were editing files with a command called ed.  Working with ed must have been an exercise in frustration – it was meant for use on dumb terminals, so in order to modify files you had to work line by line, like this:

[root@xen2 ~]# echo 'line 1
> line 2
> line 3' > file
[root@xen2 ~]# ed file
21
2
line 2
2d
2i
oh god help me
.
2
oh god help me
w
29
q
[root@xen2 ~]# cat file
line 1
oh god help me
line 3
[root@xen2 ~]#

Now just imagine editing a sendmail configuration like that without wanting to rip your eyes out.

Mr. Joy wrote an improved version of ed called, ex.  From there, he continued to build on it so it was able to use the terminal in such a way that it was a visual editor instead of a line editor.  And thus vi came to be.

Those line editor roots are still present in vi, and actually can provide great shortcuts.   For instance in command mode:
Easy search and replace of string1 for string2

:%s/string1/string2/

Remove all lines that just contain whitespace:

:g/^\s*$/d

Occasionally with files from a windows box, there might be some stray line-feed characters in the file.  You can easily remove them with:

:%s/^M//

Note: The ^M in the command actually is inputed: Ctrl-V Ctrl-M

To explain what’s happening in these commands, the initial colon begins the ex command.  Then the % stands for all lines in the file.  If you wanted to specify certain lines, you could express it like:  :1s/string1/string2/
Translated, that’s “Computer!  Please substitute string1 for string2 on line 1″.  Things that are in the first front slashes are regular expressions – meant for matching strings.  Learning and utilizing regular expressions are a key skill that makes life so much easier when utilized properly.

As arcane as some of this can be, it really is worth the effort to learn it.  These shortcuts will save you time over your career at the CLI, and it’s very likely that vi is not going anywhere anytime soon.

]]>
https://www.serverstack.com/blog/2012/10/08/the-joy-of-ex/feed/ 0