By default a regular user does not have permission to use the serial port. In my case I wanted to use two USB-to-serial adapters:

$ ls -la | grep ttyUSB
crw-rw----   1 root dialout   188,   0 May 11 11:02 ttyUSB0
crw-rw----   1 root dialout   188,   1 May 11 11:02 ttyUSB1

By looking at output you can see that the owning group is dialout. So it is a matter of adding your user to that group (and logout/login as with every user rights change). The bash environment variable $USER contains your username.

$ sudo adduser $USER dialout

Found on the Ubuntu help pages.