Design Goals Are Important
Most game designers, before they start, have a design goal in mind. It might be "make a better 40K" or "make a spaceship game without so many hitboxes." It may be to make a game of a genre no one else has made to your satisfaction (my primary motivation). Sometimes it revolves around recreating combat from a period of history (WW2) or setting (Star Wars, Conan).
Without design goals, it's just throwing sh*t at a wall and hoping something sticks. Or (more likely) rebranding your own or someone else's mechanics to a new setting (however unsuited). Without clear design goals, we have no way of thinking critically about our rule choices.
"I wanna make a 15mm sci fi game" <- very vague, will probably end up reusing current favourite rules
"I want to make a 15mm sci fi horror game of demon possessed special forces teams" <- more goals, more specific, allows you to make critical choices about which rules you should use
"...which should include drones, hacking, a demonic mana pool shared by cultists, emphasis on morale and terror, darkness & vision, ammo jams, and horrific wounds - Alien meets Doom meets Event Horizon..." <- now we are really specific! We have a very clear framework to measure rules against.
Once we have a clear game design in mind, we know what we can include and what we can prune out. Furthermore, we also know what aspects of the rules we should pay particular attention to. I'll call these "key mechanics." We have the 4 Ms - Move, Morale, Melee, Missiles - and also Initiative/Activation, Vision Mechanics. These are bare essentials, common to all rules. There's also the extras - "Chrome" - the shiny things. Perhaps a "5th Element" - like magic (fantasy) or hacking (cyberpunk). Or some sort of resource management. These "extras" are usually setting specific.
Now, we can choose mechanics. Or rather, the mechanics we will emphasize. The setting and your game design aims will inform these choices. Usually, the more effort and detail we put into a rule, the more complex it is.
Does the rule give a good decision point?
Does the rule suit the the setting?
Is the rule worth the hassle? (aka complexity/extra work/recording/memory/dice rolling)
A game about WW2 would probably have robust shooting mechanics, cover and suppression rules (key mechanics). It may not fuss much about hand to hand combat. A game about high fantasy will probably have extra rules for magic. It would probably not bother with suppression or reaction fire, and may be dismissive of morale.
By picking and choosing your focus, you can abstract/simplify less important rules and put more effort and detail into what matters. By having a clear idea of the 'feel' of your game - the way you imagine it will play, the tactics it will use - you can better optimize your rules.
Now I'd like to pause and discuss two schools of thought:
"Fit the Game Setting to the Mechanics" - this is getting an existing ruleset, bolting on a few extra rules, and using it for a completely new setting. Think 40K morphing into FoW/Bolt Action. Or Stargrunt turning into Tomorrow's War. Or pretty much everything in Wargamesvault (indie designers have little time for playtesting, so using an already successful ruleset is very appealing). Whilst the trimmings are different the core of the game is the same. The benefit is tested (presumably popular) mechanics and an already receptive consumer base who need little effort to relearn familiar rules. The design goal might be "Turn my WW2 rules into a sci fi ruleset because people asked for it."
The downside is lots of generic cookie-cutter rules and clones of popular rules, and you can end up with WW2 games that play like medieval fantasy.
"Fit the Mechanics to the Game Setting" - this is starting out with a clear setting and goal in mind, and choosing mechanics to suit. "I want an aeronef game with little recording, but orders like Battlefleet Gothic, that isn't just WW1 naval in the air, but has actual unique aerial tactics." I can now borrow from other rules (or invent my own) to make the gameplay on the tabletop match my vision.
A good exponent of this is Two Fat Lardies. They pick mechanics to exactly respesent how they want the game to play. While TFL rules usually ends up as a (deeply flavourful) mass of house rules all using different dice and mechanics - it doesn't have to be that way. We can use consistent dice mechanics by looking at the underlying percentages and what you need the dice to do (more on this later.)
There's nothing wrong with re-using other rules. In fact, I heartily recommend it. Why reinvent the wheel if you know the rules work? What I dislike is wholesale converting of rules from x to y setting, when the rule/mechanic is not suited to the genre, or better/simpler/more flavourful solutions exist.
"Steal from one person and it is plagiarism - steal from many and it is masterful research."
May I recommend regularly looking through your library of rules books from ideas? It's so easy to fall into a "type" (Frostgrave, SoBH, 2HW - they all have a set of mechanics they default to). It's not always even deliberate. Constant exposure to rules also can lead to unintentional duplication. I mean, how many games has 40K influenced - sometimes even people who were trying to escape 40K and make their own, better game?
Heck, today I went into Secrets of the Third Reich to borrow wound mechanics and found myself being influenced by their morale rules which I had NOT intended to copy. Oops!
With a clear design goal, you can pick the mechanics and rules you think best suits your game, using critical thinking - not blind copying.
Hmmm. Now I'm not sure if I was clear on a Game Setting vs a Design Goal.
I've kinda used them interchangeably above, but they aren't the same. A
game setting is a playstyle that mimics a particular style of combat
defined by history, tech, movie, literature - or your imagination. A design goal is more a wishlist or specific point. A design goal could be
"make a sci fi ruleset that allows me to use any 15mm mini." But that's
not game setting- which would be more like "Star Wars" "Napoleonics with Magic" or
"Supercavitating Jet Submarine combat" A game setting probably includes several design goals to define it.
Wow that was a big segue.... ..okay, back to "Key Rules" or "Key Mechanics" - areas of the game where you will expend more time and effort. These could be from what I'd call core mechanics common to most game - initiative/activation + the 4 'M's. Or they could be 'extras' - special rules, magic, and other shiny, setting-specific mechanisms and rules. If it is a key mechanic it is an area of the game which you particularly want to emphasize - it is important to the feel and tactics of your game setting.
Morale (as I have noted recently) is seldom a key feature of a game. Who wants to focus on people freezing or running away? It's just not fun. So it makes sense to many devs to expend little time and effort here. Despite being a core plank of warfare, they never take up more than a paragraph or two. But if you are playing a horror game, you may wish to expend more effort in this area - it may be a key mechanic.
Psychic Powers (space magic) are a part of 40K Kill Team, but they have less weight than shooting mechanics (and are not a key feature) compared to the magic in Warmachine which is much more of a central feature of the game's tactics.
You also need to consider the overall complexity of your game. I mean, in my Forgotten 15mm house rules, I wanted drones, hacking, robots as well as a magic system that underpins the whole game. That's a lot of extra bolt-on rules. The rest of the game had better be pretty darn simple or it will bog down. So movement, shooting, initiative - it all had better be pretty simple. My priority list goes:
1. Magic system is very important and underpins the game - similar to Warmachine
2. Morale is a bit more involved than usual
3. Simple but familiar shooting, melee, movement (cover is important)
Nice to Have: Dark/light vision mechanics, ammo shortages, wounds
So now I know not to make up some super complex activation mechanic, but instead keep it simple - maybe alternate activation or something basic. Likewise I'll probably keep easy, familiar 40K-ish shooting and movement - you know, roll 4+ on d6 to hit, 4+ to wound or something of that ilk. This reduces the mental overheard, if I'm going to have all these extra morale and possession rules I need to keep the rest of the game flowing. If I add detail in one area I have to simplify elsewhere; otherwise the game bogs down.
Dice Mechanics vs Dice Percentages vs The Feel of the Game
What dice mechanic best suits my game design? What suits my setting? This is something I think of only once the game is well-shaped in my mind.
Despite folk loving particular dice methods, they are just a random number generator. It's the effects that concern us - or the underpinning math that gives results. 10+ on a D20 is the same 50% as 4+ on a D6 (albeit more swingy), but when you are rolling multiple dice things are more complex.
What matters more when choosing dice mechanics is simplicity/familiarity, ease of use, and granularity (i.e. how many results you need to show on the dice). Single d6s like 40K have limitations because it doesn't allow enough variation between troop types, weapon etc - you end up with extra rules, rolls and saves etc to compensate. Another is consistency - can you use the same dice mechanic for shooting, melee, morale AND your magic system? A consistent dice mechanic is great as it is intuitive and reduces the mental load on the player.
First decide - What do you need the dice to do? Then ask: Which is the simplest/fastest dice method that does this? Finally: Can I use this mechanic consistently in most areas of my rules?
Random: While I usually dislike custom dice, I enjoyed this article about Warcaster's dice (or the underlying math behind using specialist dice; there are arguments in favour of them; simplicity and managing odds)
Ultimately dice mechanics (d10 vs d20 vs 2d6 vs buckets of dice) matter less than you'd think, compared to the underpinning math. (I've talked about lethality percentages elsewhere). For example Infinity is a game with very strong shooting and cover mechanics. Standing around in the open is well-nigh suicidal. This is because guns are very strong and have an excellent chance to kill. Cover actually gives big boosts to both 'to hit' and 'damage.' But what if I nerfed all gun accuracy -30% but removed all cover modifiers?
The d20 Infinity dice mechanics remain the same. All the rules are the same. I just changed a few modifiers. I've just radically changed the feel of the game. Now there is less incentive to camp and shoot from cover and melee is more viable. The dice system is less important than the underlying math.
Early in the design stage, I actually often merely state the % chance of something happening without specifying a mechanic i.e. infantry 50-60% to hit, if hit, 30-40% chance to wound, 20% chance to die outright. Or spells have 60% chance to work, but novice spells are +20% and master spells are -20%. This can then be converted approximately using whatever dice system you prefer.
Conflicting Design Goals
Before I head back to work, I'd like to discuss conflicting design goals. This is when two design goals hinder each other. For example, in my 15mm sci fi horror house rules, having a strong horror vibe is a core design goal. Another design goal was to play a game with lots of minis - ~16-20 per side a la 40K, working in 4-man fire teams. This is because (imo) 15mm minis look lame individually, and I want them to look cool. However, this conflicts with my other game design aims, which would work much better in an individual-models-are-their-own-unit (like Infinity or Kill Team) with only 6-12 models.
- Scary things are scarier when you are one guy, rather than 4 guys
- I want it so ammo can run out (at the worst moment), and models can be wounded and drag themselves around; both are easier to track with less minis
- Removes the 'who is effected by what' of a fire team, and arbitrary coherency rules etc
-There's a lot going on (possession 'magic', hacking, drones, morale) so less minis = less dice rolling etc
My heart says "Keep fire teams so they look cool on the table!" while my head says "This would all be more logical and easier if you went to single models." I have conflicting design goals.
A common, but less obvious conflicting goal indie developers struggle with is "make a generic rules that allows you to use each and every model in your collection" vs "make a fun, unique game/a game people actually want to play." A sign of this is when the main draw is creating/painting up warbands but no one actually plays the game for long. Or one person is really keen but no one else is; as the setting is uninteresting or the gameplay is bland. By trying to do everything it is good at nothing.
It's why Frostgrave succeeded. It came with a strong, distinct theme and setting (duelling wizards hunting relics in icy wastes) which didn't lock you in a manufacturer but it's primary selling point wasn't to allow you to use each and every model, but to deliver a deep Mordhiem-esque campaign - and it stood out from the countless generic fantasy indie rules (despite it's rather meh gameplay).
Argh - there's more to explore but I've run out of time. I've been busily painting supervising children so I should have some posts of my 15mm toys up sometime soon.
TL:DR
Having a clear design goal allows you to choose which rules to emphasize (key features) and which to streamline
Different genres have different key features (i.e. shooting & suppression in WW2, melee and magic in fantasy)
Rules should be critically analyzed based on their relevance, simplicity and the decisions they add
Usually designing the mechanics to fit the setting is better than making the setting fit the mechanics
Dice mechanics are less important than the math underlying them; they should do everything the game needs them to do as simply and consistently as possible
Beware of conflicting design goals - something has to give