Considerations of a few alternate spell systems, inspired by various OSR sources:
System I: Saving Throw Magic
Design note: This system is loosely based on Chainmail, plus various OSR emulations of it I've seen over the past few years.
A magic-user must roll a saving throw to cast a spell. The saving throw is modified by the magic-user's intelligence modifier.
Magic-users memorize/prepare and learn spells per the usual rules. When the magic-user casts a spell, the player rolls a saving throw. If he succeeds, the spell goes off as normal and the spell is not lost; the magic-user may attempt to cast it again. If the saving throw fails, consult the following table:
2- Spell fails, is lost, demon appears with HD equal to spell lv x2, hostile to the magic-user
3- Magic-user takes 1d6 damage/spell level, may make a saving throw for half
4- Random spell of the same level goes off, centered on the same target. Spell lost.
5- Spell delayed 1d3 rounds and is lost
6- Spell delayed 1 round and is lost
7- Spell fails and is retained
8- Spell fails and is lost
9- Spell delayed 1d3 rounds and is retained
10- Spell delayed 1 round and retained
11- Spell is max damage or double duration and is lost
12- Spell is max damage or double duration and is retained
Pros: Simple, lets a lucky magic-user do his thing.
Cons: High level wizards are magic-using machines, spell mishap table is pretty basic.
Suggestions: Feel free to gonzo up that chart as you see fit. Warhammer Fantasy Roleplay has some good ideas for spell mishaps. You could also plug in pretty much any table from anything Jim Raggi has ever written if you're really feeling like a sonofabitch. (Edit: you can also use the slightly crazier table I suggest in the next system)
System II: Parameter-Based Magic
Design note: This is basically cribbed from Matt Jackson
The mishap table is my own.
Spells, as written, are precisely formulated versions of that particular spell; they are a safe version of power that can be expended in a predictable fashion.
However, magic-users can go above and beyond the normal parameters of magic at their own peril.
Magic spells have four components: effect, range, duration, and targets.
When a magic-user alters the parameters of an existing spell, he accepts the risk that the spell will get out of his control. Parameters may be increased by "steps," with each step of each parameter imposing a -1 penalty to the saving throw.
Effect: Each step of effect either adds 1 die to damage, an additional +1 or -1 to the spell effect, or increases the effect of the spell by 1.5x. For instance, stepping up a sleep spell would probably increase the max HD affected by 1.5x, so a 6 HD creature could potentially be put to sleep.
Range: The range is doubled for every step. Spells that have a range of touch step up to 10 yards.
Duration: Instantaneous spells step up to 1 round. Otherwise the duration of the spell doubles.
Targets: Spells that work only on the magic-user can be used on one other person. Spells that work on a specified number of targets double the number of targets, assuming man-sized targets. Spells with an area of effect multiply the area by 1.5x for each step.
The magic-user totals up the penalties to the spell and rolls. On a success, the spell works as the magic-user altered it. On a failure, the spell breaks from his control. Consult the following table:
2- Explosion! 20 ft. radius, dmg 1d6/spell level, centered on magic-user. Save for half damage
3- Explosion! 20 ft radius, dmg 1d6/spell level, centered on target. Save for half damage
4- Target is polymorphed into a random small animal. Saving throw reduces duration to d6 rounds
5- Spell affects everyone in a radius of 1d6x10 feet, saving throws as applicable.
6- Spell becomes alive! HD= spell level, creature can make a melee attack to bestow spell effect
7- Spell effect reverses.
8- Random spell of the same level is cast, centered on intended target
9- Random cleric spell of the same level is cast, centered on intended target
10- Spell works, but magic-user takes 1d6 damage/spell level, save for half.
11- Spell works, but no magic can be cast within a 1d6x10 radius for 1d6 turns.
12- Spell works, but the magic-user can never cast it again. If he tries, roll on this table.
Optional Rule: A magic-user can hedge his bets by casting spells in a controlled environment utilizing focus implements. The following modifiers apply to casting spells:
*Magic-user's intelligence modifier
*Magic-user is using a focus item (staff, amulet, crystal, etc) worth at least 100gp/spell level:+1
*Magic-user is casting spell in a well-stocked sanctum or laboratory: +1
*Magic-user doubles the casting time: +1
*Additional magic-user assisting the lead caster: +1/caster. Max assistants that can be used is equal to lead caster's level.
*Hit points sacrificed by caster: +1/hit point. This damage cannot be healed by magic, it must be healed by natural rest.
Optional Rule: If the magic-user takes any damage or is otherwise disrupted while casting, roll on the mishap table regardless.
A note about additional magic-users: All assisting magic-users must have a focus item to get the focus bonus, and effects for a failed roll affect all participating casters.
System III: Freeform Risk/Reward Magic
Design note: This is basically also cribbed from Matt Jackson.
System III follows the basic premise of System II, that spells are malleable and crafted by magic-users. However, in System III, there are no "safe" spells, but neither are there spell slots. The magic-user still needs to roll a saving throw to cast the spell, using the chart for System II if the roll fails.
In this variation of the rules, magic-users still learn spells, but the spells as written are ideas, theories. A charm person spell learned at 1st level is simply the idea for a magical compulsion spell. When the magic -user wishes to cast it, he determines how powerful, how many targets, etc.
This system requires a bit of DM fiat. (which is apparently a dirty word to some of my readers, but whatever.) How much effect is there in a feather-fall type spell, for instance? Given that in AD&D it's a first level spell, I'd maybe say it's only the first step of effort.
The basic step chart is as follows:
Effect: baseline 1d6 or +1/-1 or the equivalent of a 1st level spell. Each step increases by 1d6, +1/-2 or 1 spell level worth of effect.
Range: Starts at touch. First step is 10 yards, then double range each step.
Duration: Begins at instantaneous or 1 round. Steps go this way: 2 rounds, 1 turn, 1 hour, 5 hours, 1 day, 1 week, 1 month, 1 season, 1 year, 100 years, forever.
Targets: Begins with caster only. First step is a man-sized target, doubling each step thereafter. (Note that exceptionally large enemies might count as more than one man-sized target) For area of effect, start with a 5 yard. radius and double for each step.
So, let's cook up some possible spells:
1. Feather Fall
Effect: Caster falls at a safe rate, 1 level of effect -1
Range: None. 0
Duration: 4 rounds (two steps) -2
Targets: None, caster only 0
A magic-user with an intelligence score of 13 and a rare feather pendant (focus item) is looking at a -1 total saving throw to cast.
2. Fell Grasp (aka Darth Vader Force-choke)
Effect: 2d6 damage -1
Range: 10 yards -1
Duration: 2 rounds -1
Targets: 1 -1
3. Unseen Shield
Effect: +4(or -4) to the magic-user's armor class -3
Range: None 0
Duration: 1 turn -2
Targets: Caster only 0. (-1 to cast on an ally)
Total: -5, -6 to cast on an ally. A lesser version granting only +2/-2 to AC would only net -3/-4 to the saving throw.
Effect: 2d6 damage to crops -1 (pretty sure 2d6 will kill most plants well over)
Range: 1 mile -8 (yes, that's rounding up. DM fiat)
Duration: Instantaneous (occurs just before sunrise)
Targets: 1 acre -9
This spell will instantly destroy an acre of crops a mile away. How can you pull off a spell like this?
Gargus the Grim has Intelligence 13 and an ashen staff carved with runes. He's still looking at a -16 to roll with this spell. He invites six other evil magic-users to his inner sanctum. That knocks it down to -9, still a hefty feat. He sacrifices 6 of his own hit points, knocking the final total down to -3.
As Blight, but does 2d6 damage to all living persons in an acre-wide area. Even those normal humans who make the saving throw are likely to die...
DMs who find the freeform system too powerful can consider this:
System IV: Domain-Based Risk/Reward Magic
Design note: This is a combination of Matt Jackson's system and the Gramarye magic system for the FUDGE roleplaying game. There's also some Ars Magica in there, as well as ideas from the old Gamecube game Eternal Darkness: Sanity's Requiem
System IV works the same as System III, but places more restrictions on what types of effects magic-users can cast. Magic-users learn the Words of Power, which allow for certain types of effects. They also learn Realms, which are what they can use those effects on.
Words of Power
Destroy- dispel, weaken, damage
Commune- communicate with
Control- charm, dominate, manipulate
Create- Note that some forms of creation might be beyond mortal ken (create spirit, for instance)
Know- read, reveal, detect
Protect- Also protect against/ward
Restore- includes healing
Summon- bring something from somewhere
Transmute- Change something from one Realm to another. Magic-user has to know both Realms.
Air - also any gas or vapor
Animal- Refers to non-intelligent creatures
Body- The physical living bodies of any creature
Death- Corpses and corporeal undead
Earth - Includes all forms of gemstones and metal
Fire- Any energy form as well (explosions, lightning)
Illusion- light, darkness, and phantasms
Magic- Refers to magic itself
Mind- Sentient minds
Plant- Plant matter or plant monsters, living or dead
Spirit- Lifeforce, souls, spirits
Time- The flow of time. Can use effects on past, present, or future.
Water- Includes ice, cold, and most liquids.
A spell to change lead into gold would require Transmute Earth.
A spell of healing would be Restore Body.
A shield spell would be Protect Body.
An anti-magic shell would be Protect Magic.
Once again, DM fiat is how you'll determine what the level of effect is for spells that aren't simply x dice of damage or x bonus/penalty.
As for what Words and Realms a magic-user has access to, I suggest one of the following methods:
1. Give them all Realms (option: elves only get animal, plant, and spirit), but they only begin with one Word and gain one Word every experience level.
2. Same, but they can steal or learn words from other magic-users.
3. A magic-user begins with 1d3 Words and 1d4 Realms. They can gain one of each every time they level, and can learn Words or Realms as they learn spells under the existing rules.
I'll ruminate on these for now...additional systems may or may not follow. Right now I'm kind of digging on I and II, but YMMV.