A very good friend of mine got rid of his SLR kit in favour of a Leica X1. He wanted a simple, light weight, fantastic quality camera and decided that for his style of shooting he could make do with a fixed lens. A few years on and he still shoots with it every day.
Some of his work:
A Photographic Road Trip with the Leica X1
His wife still owns a Nikon DSLR so he can access other lenses if he has to but its a rare occasion. I have very nearly done the same thing with a X100 a dozen times but kept getting the sweats so I'm probably going to go down the route of a XE1 with a couple of lenses later in the year.