I'm more and more convinced that the dice rolling mechanics are relatively unimportant as to whether a game is good or not. Whether you use d6 or d10, whether you add or subtract modifiers, whether you use contested rolls or a fixed target number.... ...it's actually not vital to make your game good. There are other (more important) elements to consider.
The mechanics merely need to be simple, consistent, and (if possible) familiar.
I'm not trying to controversial, just trying to be helpful. I think many game designers overly focus on kewl tricksy dice mechanics where their time could be more usefully spent elsewhere. In fact, when making home rules, the dice rolling mechanics are the last things I consider; in quite a few, I've swapped dice types and mechanics after playtesting, usually to "speed things up" when I realise I could do it simpler, with zero end effect on gameplay.
Why Dice Mechanics Aren't Important
Dice mechanics do not define the style of game. They do not help make tactics more historical. They do not make players play a particular way, or define the meta. It's the percentages that matter (lethality) and the activation (who goes first); whether you are using a single d20 or buckets of d6, you can get similar end effects. Dice mechanics don't necessarily make players play a particular way.
Move:Shoot Ratios >>> Dice
Changing the ratio of movement distance to shoot distance can change your game vastly. The "normal" wargame has units move 4-6"and shoot 24-36". The ratio of move:shoot is usually 1:4 or so, favouring shooting. This is mostly (I believe) due to tradition and commonsense impacts of a normal 4x6 game table and small amounts of terrain. However, imagine a game where units moved 1" and fired 20"(sounds like a modern naval wargame). Now imagine a game where units moved 20"and fired 1"(ancient skirmish/melee?). The two games would play vastly differently.
Modifiers >>> Dice
The modifiers for your dice rolls are more influential than the dice mechanics and dice types used.
Let's say a game has 3+ (67%) to hit enemies. But if they are in -1 if in cover: they are only hit on a 4+ (50%). But what if we changed the modifier to -3? They are only hit on a '6' (17%) which means that cover is so massively beneficial that I predict units would seldom move.
Table Setup >>> Dice
Even something as simple as table setup - making your game table devoid of cover vs buildings every 4"with no long sight lines will impact your game experience more than whether you are using a d6 or a d8. A game dev who agonises over which dice size to use but does not consider table setup or deployment rules has made poor use of his time. Even victory conditions (increasingly wargames have ways to win without "kill em all" or "scrum in the middle" can have a bigger impact.
Activation/Initiative >>> Dice
Longtime readers would know how much importance I place on activation and initiative; I was hating on IGOUGO long before it was fashionable. Activation determines the "flow" of the game; the "when" of your movement is just as important as the "where." Simply changing from IGOUGO to alternate activation will make vast changes to your gameplay flow, let alone reaction mechanics, action points/pools. I spend a lot of time on these in other game design posts so I will not rehash their importance here, though I recommend #68 and #69 on momentum and breaking up the turn.
It's the final result that matters: Lethality
At the core, it is the end percentage of success created by the die/dice rolled, rather than how you got there. I tend to look at lethality in combination with modifiers. If your percentages are simple it's actually quite easy to predict how your game will "pan out" before you even playtest.
Now I'm not saying that the topics above are the only ones to consider; nor am saying what dice resolution you use is completely irrelevant. I'm just saying it should be a long way down your list of priorities.
Best Practice: Lowering the Barrier to EntryBasically, dice mechanics should keep the skill floor low (i.e. the knowledge you need to be able to play) with very little knowledge needed. You should be able to pick up dice and chug them without much thought. Simplicity, consistency, familiarity are all good.
Basically, as this means rolls are uncomplicated as you can get. After all, dice rolling detracts from the actual "meat" of gameplay - the decision making. Unless you are using a dice pool or some sort of system where you "game" the dice, every minute spent on dice is a minute not spent on decision making or tactics. Computers can do this instantly, under the hood so to speak. But wargamers manually rolling dice take up a lot of game time. If you are spending more time rolling dice than moving minis, then something has gone awry.
Ideally, I should be able to absent-mindedly chug the dice while thinking about my next move, just noting the results at the end.
As an example of what not to do: I remember Silent Death had a dice system using d4s, d6s, d8s and d10s (even d12s and d20s). Each weapon had a different rule and even different combinations of dice. I.e. "for a blaster, roll 2d6 + 1d8 and use the highest two dice for the "to hit"; then use the middle dice for damage." But a phaser might roll 2d10 + a d12, use the lowest two dice "to hit"and the highest two dice for damage. They were so proud how they managed to combine the "to hit"and "to damage"into a single roll... ...but didn't notice they'd actually made it more frickin complicated!
This is a classic example of how trying to be overly clever with dice mechanics actually made the game worse.
I've used the example of Bag the Hun (and almost any TFL ruleset) vs say Warmachine. In BtH, the game uses seven completely different mechanics to resolve actions. That's incredibly messy and you need to remember both how (and when) you need to use a particular method. In Warmachine, you pretty much use 2d6 vs a target number in every situation. Consistency means you only need to learn one set of dice mechanics.
There's a reason games like Bolt Action and Flames of War merely uses a thinly disguised version of WH40K's dice rolling mechanics. Or why 40K hasn't changed much over decades. Familiarity with a system lowers the barrier for entry - a player instantly can grasp the "feel" of the game and there is little new knowledge needed. You don't need to reinvent the wheel, and in most cases it doesn't benefit your players anyway.
But.... what about Dice Pools and Probability Curves? : Interesting but Messy
Obviously a single d10 and 2d6 are not the same. Adding dice together (or using buckets of dice) can "smooth" the rolls, making them more predictable. The buckets of dice method (throwing handfuls of dice with say a 4+ or 5+ as a success) does something similar. But for a budding game designer, they can be a pain to balance. A +1 modifier on a single d10 is 10%. On a d6 is is a flat 17%. But modifiers on 2d6, for example, are not equal depending on your target number. Changing a +1 from a 7+ to an 8+ on 2d6 changes it from 58% to 42%. A +1 changing 11+ to 12 on a 2d6 is 8% to 3%.
The +1 modifier does not have the same "value." If this is confusing, then it's a good reason why these methods are not ideal for the amateur game designer. Whilst I like managed probability, unless you have a compelling reason to use these methods (or love math), they make game balancing/tweaking far more difficult. I do like dice pools (which can add depth to gameplay through resource management aspects) but they do add to the game's complexity/play time and "mental cost."
Anyway, I hope I've shown how agonizing over which dice to use is fairly unimportant in the big scheme of things. Focus on other stuff - starting with a mission statement/rationale aka success criteria, consider how all the elements (move/shoot, activation/initiative, table setup, deployment, lethality, etc) will combine to make players play the game using tactics you envision. Ruthlessly keep to your original goals - e.g. if your aim is a fast play game, consider hard before you add ANY complication that does not promote your core philosophy. Sometimes a cool mechanic is actually not the best for the specific game. This is especially true of dice mechanics. Think about the big picture, and keep the dice rolling simple, consistent, and "under the hood."