Emacs: replace tabs with spaces

When you want to replace tab with spaces or vice versa don’t use M-% (query-replace) but M-x tabidy or M-x untabify. They work on the current selection so if you want it to be applied to a whole buffer, try C-x h (mark-whole-buffer) before to select the whole buffer.

Posted in emacs | Leave a comment

Searching and replacing in emacs

Day two of my serie about emacs, about searching and replacing.

Function name : isearch-forward
Typical Key sequence : C-s
How to get help : C-h f isearch-forward
Usage : isearch-forward let you type a string to be searched incrementally in the current buffer, successive following C-s will jump to the next match.

Function name : isearch-forward-regexp
Typical Key sequence : C-M-s or C-u C-s
How to get help : C-h f isearch-forward-regexp
Usage : isearch-forward-regexp let you type a regexp to be searched incrementally in the current buffer, successive following C-s will jump to the next match.

Function name : query-replace
Typical Key sequence : M-%
How to get help : C-h f query-replace
Usage : M-% search RET replace RET
Then, for each term found, query-replace will ask you what to do : space or y to replace, delete or n to skip, RET or q to exit, ! for ‘yes for all’, ? to get help about how to enter recursive edit / delete match and recursive edit / edit replacement string / …

Then you should read about replace-string, replace-regexp, occur, list-matching-lines, multi-occur, multi-occur-in-matching-buffers, how-many, flush-lines, and keep-lines.

Case sensitivity :
A search is by defaut case insensitive, but if you input an upper case letter, it become case sensitive. M-c during a search toggle the case sensitivity.

Configuration variables :
You may consult the documentation about those variable typing :
C-h v variable
or
M-x apropos-variable RET case-fold-search RET

  • case-fold-search (Non-nil if searches and matches should ignore case.)
  • default-case-fold-search (Default value of `case-fold-search’)
  • tags-case-fold-search (Whether tags operations should be case-sensitive.)
Posted in emacs | Leave a comment

Numeric arguments in emacs

I’m starting a ‘emacs trick of the day’ sequence with :

Function name : universal-argument
Typical Key sequence : C-u
How to get help : C-h f universal-argument
Usage : C-u receive a numeric argument that is given to the next called function, when no numeric argument is typed, the value defaults to 4.

So today you can try :
C-u 9 C-n # that move cursor vertically down 9 lines
C-u C-k # that kills 4 lines
C-u C-u C-k # that kills 4 * 4 = 16 lines
C-u 10 n # that enters nnnnnnnnnn
You may ask, what about if I want to input 25 ’6′ ? C-u 256 can’t work … so you just have to separate with another C-u :
C-u 25 C-u 1 # enters 6666666666666666666666666

Some functions does not have the simple ‘repeating’ effect of receiving a numeric parameter, for example, running C-u C-l does not recenter 4 times your screen ! but the help page of recenter-top-bottom states that :
> A prefix argument is handled like `recenter’:
> With numeric prefix ARG, move current line to window-line ARG.
> With plain `C-u’, move current line to window center.

A negative argument to C-l move the current line to the line ARG from the bottom of the screen.

Posted in emacs | Leave a comment

nfsmount : rpc failed: 2

For those, here on the internet, asking themselves what is this f*cking `rpc failed: 2` while mounting an NFS, i’ts possible that teh response is here :

Your NFS client, trying to mount the NFS share will use RPC to communicate with the serveur, il will go like :

> PORTMAP GETPORT(Program: NFS, Version: 3, Proto: TCP)
< PORTMAP Port: 2049
> PORTMAP GETPORT(Program: MOUNT, Version: 3, Proto: TCP)
< PORTMAP Port 49066
> MOUNT MNT(Program Version: 3, Path: /srv/nfsroot/ )
< MOUNT Reply error 2, "remote can't support version #", Program Version (Minimum): 1, Program Version (Maximum): 2

You can see that the response is "remote can't support version #" and we should have found this solution in the RFC 1831 (RPCv2) stating :

  Given that a call message was accepted, the following is the status
   of an attempt to call a remote procedure.

      enum accept_stat {
         SUCCESS       = 0, /* RPC executed successfully             */
         PROG_UNAVAIL  = 1, /* remote hasn't exported program        */
         PROG_MISMATCH = 2, /* remote can't support version #        */
         PROC_UNAVAIL  = 3, /* program can't support procedure       */
         GARBAGE_ARGS  = 4, /* procedure can't decode params         */
         SYSTEM_ERR    = 5  /* errors like memory allocation failure */
      };

So the problem is you client is asking for a NFS version greater that your server runs … but if your server is running NFS v3, check a `ps aux | grep [r]pc.mountd` for :

root 1411 0.0 0.0 18808 1036 ? Ss Apr15 0:00 /usr/sbin/rpc.mountd –manage-gids –no-nfs-version 3

Did you catch the –no-nfs-version 3 ? If your server is compiled with NFSv3 support, drop the –no-nfs-version 3 in your configuration and it should work !

Enjoy !

Posted in Code, Sysadmin, unix | 2 Comments

Python: Introducing ppipe : Parallel Pipe

I’ll speak about my pipe python module so if you didn’t know it, you should first read the first aricle about pipes

The idea behind ppipe (parallel pipe) is to transparently make a pipe async and or multithreaded. As multithreading isn’t an easy piece of code for everybody, with loads of pollutions like locks, queues, giving code far away from the actual simple task you tried to do…

The idea is that one kind of multithreading can be nicely handled with a simple design pattern well implemented in python : the queue. The queue handles all the locking part so you don’t have to worry about it, just make your workers work, enqueue, dequeue, work … but you still have to create workers !
As pipe is not far away from the concept of queue, as, every part of a pipe command works on a piece of data and then git it to the next worker, it’s not hard to imagine an asynchronous pipe in which every part of a pipe command can work at the same time. Then it’s not hard to imagine that n threads can be started for a single step of a pipe command, leading to a completly multithreaded application having ~0 lines of code bloat for the tread generation / synchronization in your actual code.

So I tried to implement it, keeping the actual contract which is very simple that is : Every pipe should take an iterable as input. (I was tempted to change it to ‘every pipe must take a Queue as input’ … but if I don’t change the contract, normal pipes and parallel pipes should be mixed.), so I created a branch you’ll found on github with a single new file ‘ppipe.py’ that, actually, is not ‘importable’ it’s only a proof of concept, that can be launched.

Here is the test I wrote using ppipe :

print "Normal execution :"
xrange(4) | where(fat_big_condition1) \
          | where(fat_big_condition2) \
          | add | lineout
 
print "Parallel with 1 worker"
xrange(4) | parallel_where(fat_big_condition1) \
          | where(fat_big_condition2) \
          | add | lineout
 
print "Parallel with 2 workers"
xrange(4) | parallel_where(fat_big_condition1, qte_of_workers=2) \
          | parallel_where(fat_big_condition2, qte_of_workers=2) | add | stdout
 
print "Parallel with 4 workers"
xrange(4) | parallel_where(fat_big_condition1, qte_of_workers=4) \
          | parallel_where(fat_big_condition2, qte_of_workers=4) | add | stdout

The idea is to compare normal pipe (Normal execution) with asynchronous pipe (Parallel with 1 worker), as 1 worker is the default, and then 2 and 4 workers that can be given to a ppipe using ‘qte_of_workers=’.

fat_big_condition1 and 2 are just f*cking long running piece of code like fetching something far far away in the internet … but for our tests, let’s use time.sleep :

def fat_big_condition1(x):
    log(1, "Working...")
    time.sleep(2)
    log(1, "Done !")
    return 1
 
def fat_big_condition2(x):
    log(2, "Working...")
    time.sleep(2)
    log(2, "Done !")
    return 1

They always return 1 … and they log using a simple log function that make fat_big_condition1 to log in the left column and fat_big_condition2 to log in the right column :

stdoutlock = Lock()
def log(column, text):
    stdoutlock.acquire()
    print ' ' * column * 10,
    print str(datetime.now().time().strftime("%S")),
    print text
    stdoutlock.release()

And that is the output (integers are the current second, so the times didn’t start at 0…):

Normal execution :
           57 Working...
           59 Done !
                     59 Working...
                     01 Done !
           01 Working...
           03 Done !
                     03 Working...
                     05 Done !
           05 Working...
           07 Done !
                     07 Working...
                     09 Done !
           09 Working...
           11 Done !
                     11 Working...
                     13 Done !

// As you can see here, only one condition is executed at a time,
// that is a normal behavior for a non-threaded program.

Parallel with 1 worker
           13 Working...
           15 Done !
                     15 Working...
           15 Working...
                     17 Done !
           17 Done !
           17 Working...
                     17 Working...
                     19 Done !
           19 Done !
           19 Working...
                     19 Working...
                     21 Done !
           21 Done !
                     21 Working...
                     23 Done !

// Just adding parallel_ to the first where, you now see that it's
// asynchronous and that the two conditions can work at the
// same time, interlacing a bit the output.

Parallel with 2 workers
           23 Working...
           23 Working...
           25 Done !
           25 Working...
           25 Done !
           25 Working...
                     25 Working...
                     25 Working...
           27 Done !
           27 Done !
                     27 Done !
                     27 Working...
                     27 Done !
                     27 Working...
                     29 Done !
                     29 Done !

Parallel with 4 workers
           55 Working...
           55 Working...
           55 Working...
           55 Working...
           57 Done !
           57 Done !
           57 Done !
           57 Done !
                     57 Working...
                     57 Working...
                     57 Working...
                     57 Working...
                     59 Done !
                     59 Done !
                     59 Done !
                     59 Done !

// And now with 2 and 4 workers you can clearly see what
// happens, with 2 workers, input is computed by pairs,
// and with 4 threads, all the input can be computed at once
// but the 4 workers of the 2nd condition have to wait the data
// before starting to work, so in the last test, you have 8 threads,
// only the 4 firsts are working the 2 first second, then only the 4
// others works.

To make the creation of ppipe simple, I excluded all the ‘threading’ part in a function usable as a decorator, so writing a parallel_where give :

@Pipe
@Threaded
def parallel_where(item, output, condition):
    if condition(item):
        output.put(item)

You can see the queue here ! :-)

Enjoy !

Posted in Python | 3 Comments