Engineers Embrace Failure on the Way to Success
I'm sure you have worked on a project where you have spent days or weeks trying to make a solution work and at the end wish you had changed directions. I'm a big fan of the 'Fail early, fail fast' concept from software engineering, but I took many years to apply it consistently to electronic design.
Some professional environments discourage failure, and unfortunately, many educational settings only offer the option of right or wrong. Changing direction can be seen as a sign of incompetence, but I cannot entirely agree with that and feel it's a sign of intelligence. Admitting failure, and admitting it frequently can rapidly speed up the design process, learning process and ultimately result in a better quality product/outcome.
We can get heavily invested in a particular direction, and it can be hard after you've invested significant resources into an approach to change directions. If you wonder how government programs can overrun by millions or billions of dollars, it can be because nobody stood up and said: "this isn't working." I've certainly had projects that have been far behind and over budget because I had tunnel vision for a single solution. It would have been faster to revise the design with different components and order new boards than it would have been trying to work around deficiencies in the component selection. I find that the more rushed a timeline is for a project, the more invested I feel in a specific solution, and the less likely I am to spend time looking at alternative approaches.
We are taught first to use breadboards to try out a circuit. This approach takes the 'fail early' approach by allowing you to rapidly re-assess the design by moving some wires, adding some components and maybe trying a different breakout board. As schematics increase in complexity, the breadboard approach gets harder to achieve due to high-speed communications which are not optimal for long wires and breadboards, or the design complexity increases beyond the practical limit for a breadboard.
Rushing straight to a PCB can be very satisfying, but doesn't allow the testing of firmware to against the circuit before locking in the design to a physical medium. Once you have a physical board, it is harder to think of the schematic as fluid, so workarounds for an inappropriate component choice start happening in software. Changing the hardware design at this point costs money, time, and other resources which reduces the chance of a revision.
As a freelancer, consultant, or startup, you may find that each project is significantly different from the last. In my freelancing career, I've found myself working on an aerospace product one day, athletic performance sensor the next and then a component for a hobbyist oriented product after that. It doesn't allow the re-use of schematics very often, and each project requires significant research into potential parts/solutions to problems. If there's a time crunch, it often results in me looking through supplier pages comparing technical specs and then committing to a part within minutes. Unfortunately, this has led me to some poor choices as I've found supplier spec sheets less than accurate in the real world, or with unmentioned caveats attached to those specifications. Committing to a component without even seeing it has been a primary cause of delays and overruns in my projects.
Two tips for learning how to fail for the good of your project
If you can't prototype the entire project, prototype small blocks
If you are not sure which component is going to be optimal for the design, test it early. If you can't find a breakout board for a part online, build one yourself. I started using a Voltera V-One for this; it allowed me to create a quick throwaway breakout board made the same day I ordered next day delivery parts.
Test every option that fits in the budget, and test it with real software - if you need to use, a microcontroller dev board; otherwise give each component a workout with test equipment. By prototyping each part, you will get an idea of the real-world performance before you commit to it.
If you haven't built and tested a specific section of a schematic, and it has any level of engineering risk for you, get a breadboard out and assemble it, or make a circuit board with just that schematic. You can confirm success before continuing and change your plans if it fails. Testing in isolation makes it easier to find flaws in a design, allowing a more rapid revision of the design.
Know when to stop
I find it hard to know when to stop chasing a resolution to an issue. The more invested in a solution I get, the harder it is to know when to stop, which turns into a vicious circle of getting further invested with less ability to say the approach needs to change. If you have tried to make an approach work, and it hasn't gone as well as you'd like, immediately take a step back. Re-evaluate whether continuing to battle the problem is going to be more efficient than scrapping the design and starting again, learning from the mistakes of the first design. If you decide to continue, first determine how long an alternative solution will take to try, and how long you feel it will take to make your current approach work. If you reach the point where you thought you could make the ongoing approach function, change to the alternative solution. If the alternative solution takes longer than you first thought, it might be time to re-evaluate that solution too. Never stop re-evaluating, as Thomas Edison said, "I haven't failed. I've just found 10,000 ways that won't work."
It can be hard to say “it isn't working”, or “this isn't optimal”. Corporate culture can make this harder still. I can also be very stressful to keep battling a solution that isn't going to perform as you want. Saying a solution you have developed isn't succeeding isn't a sign of incompetence or lack of skill on your behalf, it's a sign that you want the best for the project.