Recoil is one of the main reasons alot of people either won't try rifle shooting or are reluctant to give it a go but recoil does vary. You can actually feel recoil from rimfire rifles such as the .22LR and .17HMR although I don't think anyone could consider it unpleasant. This is inevitable because for every action, there is an equal and opposite reaction. ie with a .22LR, if you want to accelerate 3 grams of metal from a standing start to about mach 1 in the space of 16 to 24 inches, the firing device is going to move backwards and produce recoil. Centrefire rifle obviously generate alot more recoil due to the heavier bullets and higher velocities involved. As well as the equal and opposite reaction to bullet acceleration, alot of the recoil from a centrefire rifle stems from the blast of gas that is propelling the bullet. As the bullet leaves the barrel, the gas behind it is still at several thousand pounds per square inch and emerges at extremely high speed.
The video click on the left (courtesy of 6mmbr.com shows the considerable recoil from an un-braked Sako TRG42 in .338 Lapua Magnum.
The lighter rifle calibres don't produce 'unpleasant' recoil. Calibres such as the .204 Ruger, .223 Remington etc are relatively light rounds and can be shot all day. As you move upto the medium calibres such as 6 and 6.5mm, recoil becomes more noticeable but is still manageable by most people. Bigger calibres such as .303, .308 and .30-06 and upwards can produce pretty sharp recoil. Of course, regardless of calibre there are plenty of other factors affecting recoil. Rifle weight makes a big difference. Firing the same round, a 12lb target rifle is going to absorb alot more of the recoil than a light 6lb stalking rifle will. Rubber recoil pads on the end of the stock can help as well. A rifle with a thick recoil pad will reduce felt recoil more than the brass plate on a Lee-Enfield, for example.
The muzzle brake
One way to combat unpleasant recoil is to divert the blast of expanding gases that follow the bullet out of the barrel. A muzzle brake is simply a device that screws or clamps to the end of the barrel and diverts theses gases to the sides. On the larger calibres, the use of a muzzle brake can reduce felt recoil by around 40%. The only down side to muzzle brakes is that even with ear protection, the shooter's hearing can be damaged over time due to the passage of sound energy through the bones of the skull. It is also extremely unpleasant to be within 20 feet of someone shooting with a muzzle brake, indeed some ranges actually ban their use for this reason. A moderator offers similar recoil reduction but in the UK, their use is restricted to activities such as stalking and pest control.