There are numerous ways to record a synthesizer, all of which have their time and place. The great thing is, only you decide the method you'll use for any given application - no conventions, no rules, just whatever works or whatever's possible at the moment. Regardless of method, get the thing recorded and mash it with some plugins if need be.
Here are a handful of ways to record your synthesizer:
1. Computer / Audio Interface / DAW
This is likely to be the most popular option, since so much of your post-processing will occur inside a computer. Recording to the computer allows you the greatest flexibility to alter or augment the audio once it has been recorded.
To make this happen, you'll need some form of DIGITAL AUDIO WORKSTATION (DAW) to record into. Be warned: there are a ton of DAWS out there, some incredibly costly and difficult to use, especially if you're just looking to record a quick jam. I'll link here later to a separate post about what to consider when choosing a DAW.
Personally, I like Reaper because it's super cheap (free to $60), very well-endowed with features, is updated often and has an active user base posting tutorials online. That being said, any reasonable DAW will allow you to record the synth, cut the piece you've played into parts, rearrange those parts at your discretion, add effects, and of course multi-track synth parts (record many different lines on separate tracks and play them back simultaneously.)
There are a handful of synths out there at the moment that will allow you to RECORD DIRECT VIA USB: (e.g., Elektron devices using overbridge; Teenage Engineering OP-1; Access Virus; Novation Ultranova; some Roland products.) Most synths will only send/receive MIDI signals over USB, and some can be powered this way as well. If you can find a synth that is described as allowing audio over USB, then you can often just connect the synth to your computer, select the synth as an input device in your DAW and shred away.
These special synths and the remainder can also be recorded via their AUDIO OUTS. You'll find these on the back of the synth in the form of either a single mono jack or a stereo pair at 1/4" (6.35mm) or 1/8" (3.5mm). Standard audio cables or guitar cables will usually work just fine to rip the sound out. But what to do from there?
The missing link is the AUDIO INTERFACE. These little boxes take your audio signal, convert it to a digital signal and fire it into your computer via USB. They can be selected in your DAW as an input device. Typically the more expensive the unit, the better the sound quality and the higher the number of inputs that can be sent over USB. Some standalone mixers and mixing boards can also act as audio interfaces.
2. External Recorder
When I was a kid computers were only good for video games (and sometimes not even for that). My first means of recording was into the tiny microphone on my dual cassette boombox, into which I would scream high-pitched renditions of Def Leppard and Huey Lewis and the News songs.
Once I learned to play a few instruments, I upgraded to a FOUR-TRACK TAPE RECORDER. These guys are total beauts. Line your synth's audio out(s) into one or more channels of the 4-track and record direct to tape (hiss included, no charge.) An external recorder like this will also allow you to overdub (record onto the same track over top of an existing sound.)
As digital options replaced tape, these kinds of recording devices "improved" to allow more tracks at a lower cost, increased numbers of effects, options for recording to disk or exporting in various file formats, and more. You can still buy new digital recorders, including pretty cheap and sweet models from Tascam, Zoom, etc.
Of course you can always find a way to record direct or via microphone into your phone if you can download a decent app to allow it.
3. Internal Sequencer
Really the only synth I can think of that truly allows you to record itself internally is the Teenage Engineering OP-1. It has a built in digital 4-track recorder that is the highlight of the device. You can rip out mini songs in a heartbeat, and it can really help you to record quick sketches when you're away from your other synths.
What I intend "Internal" to mean is via a SEQUENCER. It's kind of a cheat, since a sequencer is just recording and playing back midi events, not the audio itself. Regardless, most modern synths now come equipped with at least a rudimentary sequencer (think the korg volca series). A sequencer will let you program in a series of notes and sometimes parameter changes and play them back at will. Most will only allow you to sequence a single note at a time, but more modern synths are coming out that allow chord sequencing as well.
Sequencers are essential for live, computer-less setups. By hitting a key on a synth or hitting its start button, you can set the sequence in motion and tweak parameters by hand or move to another synth and add accompanying lines. Sequencers are also great for dopes like me who have no business playing more than simple chords or single note progressions.
4. Other Synths/Sequencers
This is just an extension of the above point. Some synths will allow you to use their onboard sequencer to control other sequencers via MIDI (I'll link to a post on using MIDI when it's written). And then there are specific, stand-alone sequencers whose only job is to control other synths. Again, these are incredibly useful when you're preparing a live set. Let it be noted that a computer and DAW can make a great MIDI sequencer for your synths, doing double-duty as sequencer AND recorder.
5. Samplers and Loopers
These beasts are getting weirder.
Although there is some software that is specifically built to sample or loop, I'm thinking more of the standalone SAMPLERS here, such as the Akai MPCs, the Elektron Octatrack, the Bastl Microgranany, the Roland SP-404, and even the Pioneer Toraiz. These machines will record a short section of audio from your synth and allow you to mangle it in odd ways - reverse, cut into tiny sections and mixed around, pitched up or down. Some samplers allow time stretching, wherein your sample will play back at the same pitch even when the sped up or down.
LOOPERS are similar to samplers, though they often come in pedal form (thanks to the guitarists.) A looper will record a sample of audio, then allow you to record another sample over top to hear both play back at the same time. They often only have one or two tracks available, but they can be pretty useful for building up chords or ambient progressions in a live setting.
6. BONUS - amping and re-amping
If you want to get crazy, don't forget about regular old AMPLIFIERS AND MICROPHONES.
The downfall of recording any synth directly via audio outs or USB is that the signal can be very clean. Even older vintage analog synths can sometimes sound sterile and dead in a mix. One solution to this problem is to run your audio into an old guitar or bass amplifier and put a microphone in front of it. The mic can be sent to your audio interface and into your DAW or any of the other methods listed above. One thing to note is that the audio signal from a synth can be drastically different from and MUCH louder than that of a guitar, so keep the volume low and watch the amount of bass or you'll risk blowing your speakers.
Don't forget about the possibilities inherent in re-amping as well - that is taking a recorded synth signal, sending it through an amplifier and recording the result. This kind of amp use can really bring a static synth sound to life.
By experimenting with different amp settings, room placements and microphone locations, you can tailor the synth sound to whatever you're looking for. It's a great method for picking up some ambient room noise and natural reverb.
That's it! Thanks for reading. Would love to hear what other methods you've come up with for recording synths.