Picotest’s Water Cooled Probe

Having Steve Sandler in this episode is such a treat! He talks about his presentations at the upcoming DesignCon 2023 in Santa Clara and; gives us a deep dive into some very complex engineering topics, including measuring the PDN Flatness and the state space model.

Listen to the Podcast:

Download this episode

Watch this Episode:

Show Highlights:

• Steve Sandler is in the running for Engineer of the Year along with Ken Wyatt
• Steve is doing a two-and-a-half-hour tutorial on PSMR, PSOR, and PSMR testing at the DesignCon. Molex and Tektronix are both participating in the live demonstration
  • He is also doing a presentation with Heidi Barnes, Bandanin, and Ben Dannan
• A lot of conferences are going virtual. The reach is undeniably great, however, what are the pros and the cons? What is valuable to who?
• Steve talks about the conception of Picotest in the US
• Picotest made the very first water-cooled probe which he will be showing at the DesignCon
• Innovative solutions can take decades from conception to fruition, Steve shares his PhD thesis from 2011 that got him ahead of the curve
• Steve deeps dive into the water-cooled probe, how it works, and the problems it eliminates
• Power supply stability is critical in space
• What does it mean to quantify the flatness of the PDN? Steve co-authored a paper with Scott Witcher which will be presented at the DesignCon
• Steve Sandler wrote a paper in 2015: Target Impedance Limitations and Rogue Wave Assessments on PDN Performance
• FACTS! Computers in Space Station are being reset every 40 minutes
• Steve stresses the importance of simulation and gives engineers a tip: “Start out with proven models and you'll get there. Get enough confidence.“
• Steve explains why it is necessary to find the “Q” to measure PDN Flatness
• Innovation could have happened earlier, Steve talks about the typical economic problems that could be hindering technological advancements
• Steve gives us a brief deep dive into the “state space model” and what it’s attempting to quantify

Links and Resources:

• Connect with Steve Sandler on LinkedIn
• Visit Picotest website
• Check out Steve Sandler's Books
• Register to DesignCon 2023
• Watch previous episode with Steve Sandler: Being Right Matters! When, Why and What to Simulate

 

Get Your First Month of Altium Designer® for FREE

 

Transcript:

Zach Peterson:

Makes sense. Makes sense. It's not just the temperature drift, it's just that there is just so...

Steve Sandler:

There's just incredible heat and people look at me and say, "What a cool pro." What? We don't worry about stuff like that. Water cooling was the right answer. We did it. And yes, I think maybe I love the shock value. I think I do. But yes, water cooled and you'll see a lot more water cooled probes from us too. So now that we've figured out how to do it, you can bet there's going to be more of them.

Zach Peterson:

Hello everyone and welcome to the Altium OnTrack podcast. I'm your host Zach Peterson, and today we're talking with Steve Sandler, founder and managing director of Picotest. This is a real treat because we've talked with some other folks on the podcast before who are colleagues of Steve Sandler. And it's a treat for me because I follow Steve on LinkedIn, have learned a lot from him over the years and I think this is going to be a great conversation. Steve, thanks so much for joining us today.

Steve Sandler:

Well, thanks for having me. I'm excited to do this.

Zach Peterson:

Absolutely. So for those in the industry that follow the show schedules every year, they probably know what time of year it is. It's almost time for DesignCon, one of the biggest conferences in the electronics industry, especially for designers and for folks like yourself who work at very innovative companies. And I think the biggest revelation I had heard about you vis-à-vis DesignCon was that you are actually in the running for engineer of the year.

Steve Sandler:

Yes. Well, I know it needs to be in the running. I guess that it's always good to be in the running. I like that, but hopefully I win.

Zach Peterson:

Well yeah, of course.

Steve Sandler:

It's a lot more fun to win.

Zach Peterson:

Yeah, of course. I mean, if I were in that position myself, I would be happy to be nominated, but I agree with you. It's much better to actually win it for sure.

Steve Sandler:

The funny thing is, I've been nominated before and every time I'm running against guys that mostly I know and mostly I like.

Zach Peterson:

Sure.

Steve Sandler:

So, Ken Wyatt is in the running this year. I'd be very happy if Ken Wyatt won and certainly I would congratulate him, but I still want to win.

Zach Peterson:

Absolutely. Didn't Bert Simonovich win last year?

Steve Sandler:

I don't remember if it was last year, but he did win. Last year, I think it was Richard Meltzer from Sam Tech.

Zach Peterson:

Okay.

Steve Sandler:

But I think Bert's won in the past and I think like me, he's been nominated in the past. And I'm very tightly connected to DesignCon, it's my show. I love DesignCon, so I'll keep doing it, win or lose, but wouldn't hurt to win.

Zach Peterson:

Sure, sure. Absolutely. And when we had talked before, as I recall, you are involved in several sessions at DesignCon.

Steve Sandle:

Yeah.

Zach Peterson:

I know it's not uncommon to see someone's name come up more than once on a show schedule as far as speaking events. I saw your name brought up more than a couple times. So what sessions are you going to be giving at DesignCon this year?

Steve Sandler:

I really tried to keep it down this year and I still ended up with five sessions. So, first I'm doing a two and a half hour tutorial on PSMR, PSOR and PSMR testing, which is something I get a lot of questions about. And especially now with the OSFP and QSFP transceivers, the 120, the 224G transceivers. They do have power supply requirements for testing, and interestingly, nobody made anything that could meet the requirements for the testing. And yet they've made the transceivers. How that all worked and exactly, I'm not sure. But we did make a modulator for QSFP OSFP transceiver testing and we're doing a tutorial on it. I'm very excited about that one because Molex is participating, they make the transceiver. Tectronics is participating with the high-speed scopes. We're doing live demos, so that was going to be a lot of fun.

Right after that, I'm doing a presentation with Heidi Barnes and Bandanin, which is a presentation of my state space average model and how it is that it gets constructed and how to use it and why we use it. And that's a really great one. I'm a co-author on a paper by Scott Witcher on a novel use of Nism, which is software that I developed. It was actually my PhD thesis that I never finished, but it was about using impedance to determine control loop stability. And Scott Witcher is doing a presentation on an extension of that. Why don't we just use that to quantify flatness of a PDN? We said PDN should be flat and they never really are, but how come we don't have a quantifiable metric for that flatness? And NISM could do that. So I'm participating in that one. And then I'm participating in a panel with  Novak and he's doing a panel on simulators and what we need from our simulators.

And then I'm also doing a  theater presentation with Heidi Barnes and Ben Dannan on calibration and embedding. And partly, we chose to do that and partly it's like, "Hey, we could really use our session here, could you do that?" And I'm usually pretty easy. So I look in the mirror and I practice this, "I'm really busy, I can't, so the answer is no." And I practice it. Look at the mirror, "No, I'm really too busy right now." And when they ask me it always comes out, "Yes, sure." So, much for the practicing in front of the mirror, but that's how it goes, and I love what I do. So, not complaining really.

Zach Peterson:

I feel you on the, "I love what I do." And I feel you on the, "Can't say no." Especially when those opportunities come up because I have been guilty of that myself and my wife harps on me on that sometimes.

Steve Sandler:

I know how that goes too.

Zach Peterson:

But when you love what you do and especially at a venue like DesignCon or some of the other big conferences in the industry, how can you say no?

Steve Sandler:

Yes. So, there you go. There have it. It's going to be a great conference. And yes, I'm very involved. And in fact, Naomi Price who runs the conference, she said that she was going through the schedule of which sessions they decided to record, for the DesignCon digital. And she said they decide that earlier in the conference or early when they see the abstracts. And she said that, "I'm not sure how I feel about it, but you're like in every one of them, you're going to get an awful lot of camera time." And I joked with her and said, "I think I need an agent."

Zach Peterson:

Nothing wrong with camera time for conferences for sure. And in today's... Excuse me, in today's digital enabled world with everyone participating in these events remotely, I think it's great that they're actually putting out some of those sessions digitally. That was done, I believe, for PCB West a couple years ago and I've been seeing a lot more conferences that would normally be in person, also have an online or remote component. So, I would encourage any of the folks out there that are listening that are interested in all of this to take a look and sign up for some of those digital sessions because I think they're great learning opportunities.

Steve Sandler:

If you look at a conference like ED Icon, ED Icon decided to go fully virtual right after the pandemic. Everybody was worried about the pandemic, what are we going to do? And I think that for people like me, it's a mixed feeling. We love the crowd, we love being in front of an audience. We thrive on it. It gives us the energy. And yet when we do it digitally, virtually, we get to connect with so many more people. And there's the trade off and the balance. I can do web calls with a dozen people in a week and I can go to one of these conferences and do a webinar and connect to a hundred people.

So, which one is more valuable? And that is also a big question, valuable to who? The people that were in the audience at DesignCon, they saw value in the fact that they were live. Me, I saw a value in the fact that I had a real audience that I was talking to. The virtual audience is saying that they wouldn't have been able to go to the conference. So, they have the value of the fact that they now get to a time, a conference that they wouldn't have been able to attend otherwise. So I think that everybody assigns value differently and everybody gets value from it. So for me, I'm thrilled that it's out in both formats.

Zach Peterson:

Yeah, yeah, me too. I started doing the ED icon events myself right after it went virtual. In fact, I think the first one I did was 2020 and that's when it was all virtual. So, it was a big shift for me to have to, number one, learn how to do screen caps and do them in a compelling way because some of that stuff is pre-recorded. But then get used to the fact that you're not actually right in front of people. But it's kind of... Screws with your mind a little bit because there could be 200 people on a session watching you.

Steve Sandler:

Yeah, exactly. And you got used to talking to a webcam instead of a room full of people, which is an adjustment for sure. But I think that through the pandemic, like it or not, we've all learned how to do that. Not only have we learned to do that, but the mediums, the actual software's improved because we needed it. And so I think everything has kind of progressed because of it. And we are where we are, and I'll do the digital events and I'll do the live events and I'll enjoy them both.

Zach Peterson:

Yeah, yeah, definitely. So, your company PECO test, I'm familiar with PECO test, but some of the folks in the audience may not be familiar with PECO test. Could you maybe tell us what your company does, and what are the products that you guys produce?

Steve Sandler:

Sure. So, I founded PECO test in 2010. I was actually retired and I got a little bored, I got remarried. My wife went back to school and I was like, "I need to do something. I'll build another company." And I decided to build a company that does test instruments, basically test support. We had met a company called PECO test in Taiwan and they make benched up products like power supplies and function generators and things like that. They're very high quality. We were looking for somebody that would manufacture the things that PECO test designed and we asked them if they would manufacture our products. And I asked them why it was that they aren't known in the US and they said they don't really know, they just couldn't get a presence. And so I said, "I'll tell you what, if you give me a trademark to your name, I'll use it and if nothing else, you'll get the name recognition in the US."

And we got permission. We even managed to get the PECOtest.com, which pissed them off a little bit in Taiwan because they would've liked that for themselves. But we bought it. And so we think that there's this mutual benefit. We decided that we were going to make the products that nobody really wanted to make because they're low volume. And so for example, you could go out and buy a Vector Network analyzer from keysite. You buy their 5061B Gold standard instrument for doing bode plots, and yet they don't make an injection transformer. So where are you supposed to get this injection transformer from? You just kind of left to figure that out.

Same way, 5061B, it does two point impedance, but you're required to come up with a powers divider and they didn't make one for that instrument. Yes, you include one together using one of their 18 gigahertz power dividers, but they didn't make it. So we said, we're going to make a thing that plugs into the 5061B and you got to connect all of these instruments somehow to the board. How do you do that for PSR testing, for noise testing? People don't make these probes. In fact, the best in class was the P6150 probe from Tectronics. They discontinued it because it was low volume and we said, you know what? We're going to leapfrog that. We're going to make an even better probe than the P6150 and we don't care that it's low volume.

Zach Peterson:

Now Really quick. Those probes, I get that they're low volume, but are the probes that are needed just spanning up to such high frequency that they become a little too expensive and because they're low volume, the economics don't make sense?

Steve Sandler:

I think that's partly true. I think that it is part of it that's economics and the fact that PECO test is a tiny little company and that for the most part, I get to do what I want. I don't really have to worry about economics, and I never did. Even from the day that I started the compass like engineers need this, I'm going to make it. And somehow we'll make it make sense. We finally got to a point last year, I think where the economics finally started to work for us. We were able to spin our versions of our probes. We make semi custom probes for many, many companies. It was like Broadcom came to us last year. They needed a 20 mill spring tip probe. Nobody makes a 20 mill spring tip probe, but we said we can do that. They're measuring  loop power rails and they're saying the environment near our BGA is so noisy.

Could you make a filter for your probe so that we only see the PLO band from, say one megahertz to 30 megahertz? Yeah. We built band pass filters into our probes and we delivered that to Broadcom. And they're like, "Well, that's awesome." So yeah, we added this 20 mill probe. We have customers that came to us saying, "Hey, can you make a 4.2 millimeter probe?" Sure. "Can you make a 1.8 millimeter probe?" Sure. Whatever you need, we'll make it. And if you need two, that's great. We'll added to our product line if you needed it. Other people probably did too. And then we kind of expanded to higher tech things like, how do you do the PSNR testing for QSFP trans receivers? That's not for weenies. That's a very difficult task. It was a very specific modulator and it led us to make the very first water cooled probe.

I don't think anybody's ever made a water cooled probe before, but we had to move the modulator into a probe tip and we had to figure out how you get 20 watts out of this probe tip. And we decided the best way to do it is water cooling. And so we made the very first water cool probe, we're going to show it at DesignCon. And we've had a lot of interest in that. There's a lot of people that needed it. They just didn't know they needed it.

In fact, last night I got an email from, I think it was Arista, and I'm looking at their requirement and I said, "You're doing a QSFP test?" And he said, "Yeah. How'd you know" I said, "I recognize the table. I've seen it probably a hundred times in the last month. This is the modulator. You need to do what you want to do. And yes, we make it. Come see it at DesignCon." And that's kind of the way that we've done business. We kind of meandered our way through it, solving customers problems and figuring out how to do it and maybe even trying to guess what problem they're going to have next so that we'll be ready when they get there. And good, bad or indifferent, we're a small enough company that I'm allowed to do that or just have fun and the economics. Well, in the end it all comes out somehow.

Zach Peterson:

That makes sense. And I think for you to go around solving those problems in innovative ways, it builds a ton of credibility for not just you but for the company and for the brand.

Steve Sandler:

And I think that a lot of things that I've done over the years, they take a long time to come to fruition. My PhD thesis, and I never finished my PhD because I wouldn't publish the math behind it, but it was this non-invasive phase margin solution where we could look at impedance plots and tell you what the stability of the control loop was. And just now... I mean we pitched it in 2011 to Tectronics and it was almost no interest. And this year you're going to see it in Tectronic scopes. It's like-

Zach Peterson:

Really? That's interesting.

Steve Sandler:

How far we've come. NISM is available for almost every instrument right now from the Bode 100 to the Copper Mountain instruments to keyside instruments, to Roddy instruments. It took 10 years, but now it's available in every instrument and it's available to engineers because I was willing to take the gamble and wait 10 years. That's how it happened. And 10 years ago when I did it, everybody told me you're just ahead of the curve. And that's true, but it's where I like to be.

Zach Peterson:

Well, for someone in your position who is developing these innovative solutions, I think you have to be ahead of the curve and you almost have to be able to see what's going to happen in the industry in one year, three years, five years, however many years down the road.

Steve Sandler:

I think that's true. And I've done a couple of webinars, "What a scope's going to be in five years?" And of course nobody knows, but I have a crystal ball just like everybody else does. I think one of the things that happens is that, I mean, I always said, you have to pay your dues, no matter what it is you do. Somehow you got to pay your dues. I wouldn't know how to make a water cooled probe unless they had done the predecessors. I needed to know what was in the way. And you learn that by these incremental solutions. If I hadn't made these earlier versions of the modulators, I wouldn't know how to make this crazy modulator for QSFP. Now, so the only reason I can do it is because I paid my dues over the last 10 years figuring out how to make modulators and what's in the way.

And so when I got to this new QSFP requirement, it was like the only way you can do this if you can get really, really close to the transceiver, and the only way I can get really, really close to the transceiver is to put it in a probe. Except this lot of power participation in this thing. How do I get it out? Well, it needs to be a probe, so it needs to be small. So air cooling, it probably ain't going to work. And so right from the start, we knew this would be a water cooled probe.

Zach Peterson:

So is the issue there temperature drift in the modulator and then that impacts accuracy, and you would either have to compensate for it with a direct temperature measurement or you just eliminate it through cooling?

Steve Sandler:

There's two different... Or maybe three different problems. One is, by definition, because I need to get this high speed thing to get through these uncontrolled low impedences, it has to be really close. I need to get the inductance of that interconnection really close to zero. And that means it had to be an a probe. The modulator itself is a gallium nitride element. So we do want to try to keep that temperature stable so the resistance doesn't drift and the voltage don't drift and all of those things.

But there's also an incredible amount of heat, there's about 20 or 25 watts dissipated in this tiny little gallium nitride element that's sitting in the probe jet. How do you get the 25 watts out? You got to blow a lot of air through it and air cooling of something that small is really inefficient. Or you make a manifold to attach to this tiny little gallium nitrate element and you pump water through it to keep it cool. And so that's what we did. We put it in a probe tip so you can get really close and we put water cooling in it so that we can keep it cool enough to run.

Zach Peterson:

Makes sense. Makes sense. It's not just the temperature drift, it's just that there is just so-

Steve Sandler:

There's just incredible heat. And people look at me and say, "Water cooled probe? What?" We don't worry about stuff like that. Water cooling was the right answer. We did it. And yes, I think maybe I love the shock value. I think I do. But yes, water cooled and you'll see a lot more water cooled probes from us too. So now that we've figured out how to do it, you can bet there's going to be more of them.

Zach Peterson:

Very cool. Very cool. I want to go back to the non-invasive stability just for a moment, if we could. So earlier you had mentioned that it is a measure of, I guess, two things, right? First is stability of the control loop, but then there's also... You had also brought up PDN flatness. So on that first point, stability of the control loop. Are we talking about ensuring that there's a limit cycle that you arrive at that has sufficiently low amplitude in terms of an oscillation or that you essentially have enough decay in any power delivery in terms of any oscillation that you are within, whatever noise margin you need to be in for your signals that are being pulled through the power network?

Steve Sandler:

So it's really both of those. And a lot of my work is in the space industry. And you look at these new voltage regulators, a lot of them are even , I got one chip that's got a bunch of power supplies in it, and all we get to see is the output cap. But in the space world, we're really concerned about the stability of the power supplies. We've got to make sure they're not oscillating. We've got to make sure they have good responses and we have requirements for it. End of life, 30 degree phase margin, 60 gain margin. We have very specific requirements. But if you can't measure a bode plot, how do you do that? There are people that have written about the relationship between Q and phase margin. In fact, Erickson  in their book, Fundamentals of Switching power electronics, they actually did derivation of the relationship between Q and phase margin.

Unfortunately they did it from an internal path of a series loop. And we can't access that from the outside. We can only access the impedance measurement, which is a different Q. And so my goal was to figure out, how do I determine Q from outside to power supply accessing only a capacitor? And I did that. Once I did that, how do you get from that Q to stability margins? And how do we get from stability margins to phase margin? And that was my thesis. And I didn't finish my thesis for two reasons. One is that I was going to university in England and they closed their control loop course. I would've had to pick a different topic. But also because I wouldn't publish the mathematics behind NISM.

But NIM was based on, let me figure out what the Q of this measurement is. And once I have the Q of that measurement, I can use that to determine stability. We get the flatness of a control loop. It's actually a little bit simpler. We need to know what the Q was. We don't actually need to transform that to anything. We just need to know what the Q is. And so we can use Q to relate to what does that mean in terms of flatness of my PDN? And that's the work that Scott Witcher did. And Scott Witcher is doing a paper at DesignCon, which I'm a co-author of, that looks at how do we quantify flatness of a PDN by using that measurement.

Zach Peterson:

So couple things here. When you say flatness of the PDN, is this inclusive of the board and the chip or just-

Steve Sandler:

Board chips, decoupling caps, everything. And so if I go measure near the asic, how flat is that impedance? We keep talking about the fact that impedance has to be flat and it's a decent question to ask how flat is flat? When you say flat, what does that mean? How do I know if I meant flat? Flat is a Q measurement. If I look at Q, you'll tell me how far I am from flat. We may not know what a reasonable requirement is. We kind of defined it. And I think Scott Witcher may mention this in this paper, we said, "You should have a Q that's less than two." And where did we come up with two? It wasn't entirely arbitrary. We say that in stability of control loops, we're allowed to have a 30 degree end of life stability. If you work through what that means in terms of Q, 30 degrees is a Q of two.

Zach Peterson:

Okay.

Steve Sandler:

60B gain margin is approximately a Q of two. So if we're saying that a Q of two is tolerable for stability, then a Q of two should be tolerable for a PDN. And so we're going to throw a number out there and we're going to say, "You should always have a Q less than two." Maybe some will argue that's too conservative, maybe some will argue that's too restrictive. But we threw down the gauntlet and said, "That's our definition. You should have a Q better than two." Scott Witcher is saying whatever it is, we can quantify it. We can say that we looked at this PDN and it's got a Q of one and a half, it's got a Q of three, whatever it is, we now have a quantifiable number using NISM. NISM gives us that number. And so assign what you want to. It is the limit, but the methodology of NISM actually as an interim measurement tells us about the flatness, and we should use it for that. And I think it's a good paper. Scott did a lot of work on it. I think he did a great paper.

Zach Peterson:

So now I have two questions to follow onto this. So one is about target impedance and then another is about an alternative metric for flatness of just a complex function, let's say in the frequency domain. So let's talk target impedance first. So a lot of literature that you might read on PDN impedance talks about a target impedance. And essentially as long as you are below the target impedance, you're good. That's usually what the guideline kind of boils down to.

Steve Sandler:

Also, stop there for a minute because I did a paper for DesignCon. I believe it was 2015 on target impedance and road waves.

Zach Peterson:

I remember that, actually.

Steve Sandler:

So, one, it was very controversial, but one of the things that I said is that target impedance doesn't work. Target impedance only works if the PDN happens to be flat. And the example that I used was to create a road wave. I said, so let's create this target impedance. I think it was 120 milli and I'm going to have three peaks, three resonant peaks. And they're all going to be below that 120 milli. In fact, they're right at the 120 million limit. And then now, okay, so I'm at the target impedance requirement, let's see what transients actually look like. And I created this road wave that was a monster thing. And I said, "Okay, so that didn't work." And so if you wanted to look at target impedance, you'd have to say, my target impedance has to be below the summation of all the resident peaks in a PDN.

So if I add up all of the resident peaks, they have to be less than target impedance or target impedance has to be flat. And if you meet the flatness requirement, target impedance works great. If not, then you need to be below the summation of all of the resident peaks. And so that becomes very cloudy for people to deal with. Now how do I know how to set the target impedance? Because I really need to see how many resident peaks I got first. And yeah, it's complicated. It is. But I would recommend everybody go look at that paper on road waves. In fact, when I gave it, I got an email from a guy at  semiconductor, I think it was, that said, it's easy to do that in a simulator, but in real life you couldn't do it.

And I had a bunch of on-semiconductor DDR termination regulators here, and I measured one and I sent him a picture of a road wave on his DDR termination board. And there's a sticky note on it that said six minutes. And he wrote back to me, he said, "What is six minutes?" I said, "It was six minutes from the time I went into my lab until the time I came out with the picture." So you're saying it'll be really difficult to create this in a lab. And I'm saying that I did it in under six minutes and then that's from the time I walked in the lab till the time I walked out with the picture. Six minutes, not so hard.

Zach Peterson:

So just to back up for a moment, for folks who may not be familiar, so rogue waves, essentially pseudo random or unpredictable I guess would be the best way to describe it. Yes. But essentially resulting from a super position of oscillations that are generated or correspond to different polls in the PDN.

Steve Sandler:

Correct. And I'm the first one to write papers about it. Dr. Chang at San Diego University had done a paper even before I did. You're right, they're extremely rare and they're very conditional. So, in my paper what I showed was three different resident peaks and I showed the mathematics of how it is that they work. And yes, you would have to get this waveform that created exactly the right digital pattern at exactly the right times to get the superposition.

But if you achieved the superposition, if I remember right, we went from a step response of 43 milli , I'm getting this off the top of my head. I think it was 43  and the road wave was roughly 570 . So this crazy wave was 15 times what we would've expected from a step. Could it happen? Of course. Is it rare? Of course, I think Heidi Barnes said, if you look at a gigahertz CPU, if your odds are one in a billion, it would happen every second. Okay, so your odds are lower than that. But let's give this some perspective. We're creating a billion patterns a second, not so hard to come up with one that would create a road wave.

Zach Peterson:

Well, at that point it's statistically guaranteed that you will exactly generate one of these within a reasonable amount of time.

Steve Sandler:

Yeah, I did a paper or presentation for aerospace who does the space power workshop and they told me that in space they actually track likely occurrences of road waves, and they can't prove that it was a road wave. But they have these anomalies that they've seen that they have associated with road waves and they actually track that. I was really impressed by that.

Zach Peterson:

Very impressive. And that's something I would not have known that they actually need to track. But I suppose it makes sense because if you have a device up there and let's say one of these roadway waves causes a device to shut down or possibly even fail, it's not like you can just head on over and replace something or fix it or start measuring it or start probing it or whatever you need to do to figure out what's going on.

Steve Sandler:

It's just a matter of getting the timing right or the timing wrong depending on how you want to look at it. But if the timing lines up, it's going to happen. I'm actually going to show a simulation in the panel to discussion with this on Novec, which was my first simulation of the space station. It was the space station going through Eclipse, which of course it has to do every 40 minutes. And every 40 minutes, that means you're transferring the power bus from batteries to solar power to charging. And that happens every 40 minutes. And they did one simulation of it showing that the occurrence of that would likely result in the power bus falling below the reset limit, at which time everything on the space station would reset. Imagine resetting all of the computers on the space station every 40 minutes.

Zach Peterson:

It's every 40 minutes. Yeah.

Steve Sandler:

And then I showed a picture of what it actually looked like after they ignored me a year and a half later when they actually got the data and it looked exactly like my simulation. And yes, it did reset all of the power electronics and it actually had a little sticky note on it that I'm showing in the DesignCon presentation that said, "Oh crap, can you fix it?" The answer was obviously, "Of course I can, but my rate just went up."

Zach Peterson:

If there's any excuse to raise the rate, I think that would be it.

Steve Sandler:

There you go. You should not have waited a year and a half. You should have believed me when I told you the first time.

Zach Peterson:

Yeah, now it's going to cost you.

Steve Sandler:

Exactly. Exactly. And it just goes to prove that simulators can be right.

Zach Peterson:

Oh sure, sure. I mean as long as you know how to avoid the gigo, right? Garbage in, garbage out. Yeah, certainly very useful.

Steve Sandler:

Start out with proven models and you'll get there. Get enough confidence. I often tell people that the reason we spend so much time simulating is so that when it matters, we get it right. And I think about it, that the plane that landed in a Hudson, that Scully amazingly landed, I think he's the first one that ever actually successfully pulled off that landing in water. How many hours did he train to be able to do that? One thing that was so rare that... And that nobody's ever pulled off before, he spent all of his time in the simulator preparing for this event that would likely have never happened. And the one time it did happen, he was able to do it. And that's the same thing with our simulators. We do it so that when it matters, we can count on getting the right answer, that's why we do it.

Zach Peterson:

Absolutely. Absolutely. So just to return again to PDN flatness. So, one measure that you could use to determine the flatness of a curve, generally whether it's a real valued function in frequency domain or if it's a complex function in the frequency domain is something like an LP norm. So, an LP norm is essentially equivalent... Well, when you do LP equals two, that LP norm is equivalent to essentially RMS error. So why not use something like that from a direct PDN measurement to determine flatness? Why do you need to go and look at Q as you've described it?

Steve Sandler:

The answer should probably could, right? But in the Q we're looking at that deviation, but we're also looking at how fast we approach that deviation and how fast we go through that derivation determines how large the peak is. We could take an average RMS error and that can be a very narrow, very large excursion. It results in that error. Or it can be a very broad wide band error that's much smaller, but you'll get the same RS value and they're going to get different responses. So, Q uniquely gives us that derivation, but it also tells us how fast the derivation occurs. One thing that I found really interesting when I started talking about this is that it's really different depending on the engineer you talk to. When you talk to a PDN guy, power integrity guy, he's used to looking at vector network analyzers.

We do two port shun through impedance. We're looking at impedance profiles. Now you go over to power electronics guy and he's like, "I don't understand. Didn't you see this on a scope?" What does that look like in time domain? It's one of the things that we're doing very often in... These days in power integrity is this transformation back and forth between impedance and time. Because the power electronics guys don't really understand impedance. At some point they will. In fact, I'm giving a tutorial at Apec, and so hopefully we'll broaden the horizons of both the power integrity guys and the power electronics guys to bridge that gap a little bit.

Power integrity guys don't really understand power electronics. Power electronics guys don't really understand power integrity. And when we look at things, we use different tools to look at them. So how does that support our end-to-end simulation? We need to be looking in the same way, but one of the things that we do right now is this constant transformation back and forth between time and frequency. So we can satisfy the guys that are used to looking on scopes and also the PDN guys that are used to looking at VNAs. And that to me is really interesting.

Zach Peterson:

Well, I think the simulation person who is more of a mathematically inclined person would have no problem going back and forth between time and frequency because they're going to essentially look at the PDN impedance as a transfer function and just using some sort of phenomenological model for whatever that current draw is from the PDN, they'll be able to get a voltage wave form. And they'll know essentially what's going on the power bus. But would you say that also ignores what actually happens in the regulator to try and ensure or provide control and stability for the power that's being delivered into the PDN?

Steve Sandler:

It does, but also go back to my road wave example. So there is this digital pattern you could create that's going to give you this roadway, but how would you know what that digital pattern looks like? It'd be really difficult to try to figure that out.

Zach Peterson:

And then with that digital pattern, see I didn't even reference it as a pattern. I was just talking about one current spike. So, if I have one bit that I draw power for from the PDN, okay, I can predict what that oscillation is. But now let's talk about a pseudo random bit stream. That pseudo random bit stream, how do you know what that's going to do? I mean, that's much more computationally intensive for someone who does simulations trying to predict what a road wave would look like.

Steve Sandler:

So think about the differences between what we see under  scope and what we see in a vector network analyzer, and I talk about this a lot in my papers. A vector network analyzer, we see a log frequency display, we see a log y-axis. We can see tremendous amounts of data in that one screen. Now taking scope, which has a linear timescale and it has a linear Y scale. So I do this very fast transient, and I'm looking at the response of the control loop. So I have this very long measurement and the spike that I wanted to see this high speed transient was in the first a hundred nanoseconds of that measurement. And so it becomes really difficult to find what you're looking for in the scope unless you know where to look. So, one of the things that we do is we set up different windows that have different zooms, so I can see what that front edge looked like and also what their time response looks like.

This is really interesting too because when you look at the stuff that Heidi Barnes does in simulation, and I work a lot with Heidi Barnes, her time domain pictures have log X scales so that she can actually see that short term thing and long term thing. We can't do that in the scope, but in the simulator we can. And yes, the Y scale is also linear, making it really hard to see different amplitude responses. If you said of her big response, you can't see the little ones, if you said of her little ones, the big ones are off screen, so Heidi does her simulation, she can do log scale time domain plots. And it looks a little bit odd at first because you're not used to seeing that. But eventually I think, I mean it's one of my things that I say about what happens to scope in five years and five year scopes will be able to do that because they have to.

Zach Peterson:

Sure, sure. So looking on the log scale like that, you would then be able to say, identify the fastest transients that are associated with one of the polls in the PDN, and then by looking at that one poll in the PDN, you could say, "Hey, I want to attack this pole by changing my design." And you could then make some determination as to what the most appropriate approach is to do that.

Steve Sandler:

Absolutely. Absolutely. And when you see that very fast first transient, right? It is really fast and it's really narrow, then you start to think about, well, I can also see the ringing from the control loop. I did this step and I see the spike and also see the ringing and now road waves start to make sense. What would happen if I could move that spike over so that it occurred at the same time I had that dip from the control loop? Could I do that? And if I could do that-

Zach Peterson:

Oh, so that they can cancel each other.

Steve Sandler:

They could either cancel each other or they could superimpose, right? It could go either way-

Zach Peterson:

Depending on face- 

Steve Sandler:

Make sense if I just move that over, so it occurs during one of those dips, holy cow, now they've become additive and it got this dip that's twice as big or three times as big. Could that actually happen? And it's the whole thing behind. What does it mean to make a road wave? It means just moving those things along the X-axis so that they occur at the right times or the wrong times, depending on how you want to look at it. Frequency domain, it's really easy to see that because we could see these resonances in this log scale. And if you could do that in oscilloscope, you'd be able to see that just as well. The reason we don't is because the oscilloscope has a linear time scale and a linear y-scale.

Zach Peterson:

Sure, sure. I understand. I understand. That's extremely interesting though because you've already picked out the next innovation in scopes and it seems like it's something that should be so simple to do. I mean, you can put stuff on log scale on any computer. I mean, hell, you have data on your phone and if you have an app on your phone that's visualizing data, you can put in the door log scale

Steve Sandler:

For sure. I mean, you can put it in a simulator, you could do it in a scope. And these scopes today, I mean these scopes have incredible horsepower. My tectronics MSO6, I mean it's a monster thing with incredible graphics capability and incredible processing capability. So, could you do it? Of course. Why not? Because not enough people have asked for it.

Zach Peterson:

Really it's that simple?

Steve Sandler:

Yeah. Why haven't people asked for it? Because they didn't know to ask for it. And so you end up with this circular reference and that's how I keep ending up 10 years behind the curve. It's because I know we need to be able to do this. Nobody else does. I convince a bunch of people and 10 years from now they go to tectronics and say, "Whoa, we need this thing to do log displays." And tectonics says, "Wow, there's a lot of people asking for it now. I think I should do it."

Zach Peterson:

It's amazing how the demand has to pull on that to get that to happen, when it's something that should seem so simple, why didn't they just include it as an optional feature?

Steve Sandler:

So, this goes back to my company. I don't have shareholders to answer to. I don't have investors to answer to. I can do whatever I want. We get to be a tectronics, they have investors to answer to. So it really comes down to cost and return on investment. So the question always is, if I spend the money to do this and it does cost money for the programmers, then what is the return on investment? It has to have a return on investment. For me, I don't have to have a return on investment. For tectonics, they do. If I can go poll a bunch of customers, I go to get Apple and Google and Meta or whatever to say, "Hey, we need our scopes to have log displays." You can bet the tectronics would do it.

Zach Peterson:

Makes sense, makes sense. And then of course the rest of them are going to follow along and try and get some of that customer base.

Steve Sandler:

Yes. In the meantime, in ADS, it's pretty simple. Most scopes you can export the data is Excel. And if you want to do it in Excel, you can make that display log. So, it's not that you can't do it, it just takes work.

Zach Peterson:

Well, it'd be so convenient to be able to just do it right there on the scope instead of having to stream the data over to a computer and then you have a data logger on your computer doing it. Or like you said, I got to export the data, bring it into Excel, convert it, look at it, make a decision, go back to the scope and do it again where it's just much easier to see it all live.

Steve Sandler:

And we are starting to learn that. I mean, why is it that oscilloscopes now can do bode plots and they can do PSOR measurements and now we can do impedance in oscilloscopes. And very soon tectronics is going to show they have NISM in their oscilloscopes. What's the motivation there? The motivation is I have this one box sitting in front of me. It's a very expensive box, takes up a lot of room on my bench. I understand the gooey, I learned how to drive this thing. Why can't it do everything. That's why. That's the single box solution. That's what engineer are asking for because there's not a reason that this box can't do that.

And they're right, it can. You just need the impetus and you need the desire? And I'm pushing really hard on it. And tectronics is one company that's really responsive to it. So when it comes to adding these new features to their solo scopes, even TDR, they're like, "All right, you want to do this? We'll do this together, let's do this." And so we have the 10 gears, TDR near scope, we have the impedance measurements. We'll have NISM, we have bode plots, we have PSR, this very little that we won't be able to do in that one scope.

Zach Peterson:

Very cool. I mean, I guess it eliminates the need to have a very cool looking rack full of instruments if you just have a one box solution.

Steve Sandler:

I was at one company, a big, big computer company, and they told me that they love our Bode 100, but they also hate it because it's so difficult to get to. And I said, what do you mean? And they said, "All of us have an oscilloscope on our bench. It's dedicated. It's mine, it's always there. And when you buy something like a vector network analyzer, it becomes a shared resource. It goes into community pool. You got to go sign it in, you got to sign it out. Maybe it's available, maybe it's not. Then I can of carry over to my bench and then I got to get it all connected and the USB cables and all this stuff becomes a pain." And I was like, "Really? That's the problem you have with the bode 100? Is you have to go get it? And he said, "Yeah, we all had them dedicated on our bench. We love it, but now it's a shared resource."

And I was like, "Wow, that's nuts." We had a company that asked me if I could do a bode plot with less wires connecting. And I said, "Why?" And they said, cause if you look at the wiring in our bench, it's really, really crowded. We have room for two wires, but we don't really have room to snake through full wires." And so you want me to change how we do bode plots so that you can account for the fact that you got no room on your bench? And essentially that's the answer. Yes. So, my job is to allow them to make the measurements they need to make, that's my job.

And so, however it is, I do that. I keep pulling rabbits out of a hat and I figure out how to let them make the measurement they want to make, the way they want to make it with the limitations they've got. And very often that's kind of compressing us to this single box solution. They have it on their bench, they're asking to spend a lot of money on this instrument, they need to justify it and all of these things help them justify that. And that's great if that's what it takes. You bet.

Zach Peterson:

Absolutely. We're getting a little up here on time, but before we run out of time, I actually wanted to ask you to maybe jump into the state space model that you mentioned earlier and maybe if you could explain what this is, what it's attempting to quantify, things like this.

Steve Sandler:

Okay, I can do that and I can do that relatively simply.,I think. And this model just from perspective, it's 30 years old. I wrote this model and before 1990, I published it for the first time in one of my books in 1995. And the goal was really simple. How do we do fast simulations in the frequency domain of a power supply? If I simulate a switching power supply, I can turn switches on and off. How do I do an AC analysis of that so I can measure, for example, impedance? I really can't. And if I could, it would take a really long time to do it. So, state based models say, let me write an equation that tries to synthesize what the power supply does. And a switching power supply, there's essentially two states. Either have the top switch on or you get the bottom switch on.

And so if I figure out, "Well, how long or what percentage of the time is the top switch on, what percentage of the time is the bottom switch on?" And I could just average those. In simple terms, let's look at what that means, to take a five volt input to these switches. And don't want to put out one volt. 20% of the time the top switch is on. 80% of the time the bottom switch is on. And for the average that I get 20% of five volts, I get one volt out. Perfect. Now what if I take those state space equations and I try to figure out what that looks like If I put a small signal on top of it, I just modulate this duty cycle, a small amount, and what would that look like? And so the state space models a set of equations that do exactly that, and they're not magic.

They're published in my book, come along 2006, 2007 I think it was. National Semiconductor says, "Hey, we want to do simulation of power supplies over the internet." And of course, I mean, we didn't have gigabit internet then, but they want to do time remain simulation over the internet. I said, "You're crazy." And they said, "Well, there's got to be a way. You're the genius figure out a way." And they did. And I published it I think in 2008 with national semiconductor. I said, so you have the state space model that figures out what percentage of the time the switch is on and what percentage of the time the switch is off. So what if I just run that which is blazingly fast and instead of trying to solve voltage in current equations, I say, "Look, the state space model's really fast and it's smart. It knows what the duty cycle is."

So, let me just tell that why you turn the switches on and off. That's what we did. We ran the state space model. We had this overlay that said, "All right, let's just make the switches do what the state's space model does and we'll end up with this really fast time domain result." And we did. We published that, I think it was 2008. I think it wasn't until 2015 when I became a certified expert in ADS. I said, "You know what? I can do this in harmonic balance, which is a native Fourier simulation. I can do it much faster." And since it's native Fourier, it can also get the spectral content. Of course, Heidi Barnes comes along and says, "Well, why not just do EMI too? We can get EMI out of it." And so there's state space models doing all these things that are never intended, but we can do all of these simulations now.

We can do a simulation of time remain response and PSR and bode plots, spectral content, all that stuff. And the entire simulation runs in under a second. It's crazy. And I said, to create this model, the equations exist. We just need a couple of terms in the equation. And it just turns out that there are terms that for whatever reason, the power supply manufacturers don't provide. And I don't know if it's because they don't know that we need that number. I don't know if they think that it's secret, but it's said, "All right, let's say that it's a secret. I'm pretty good at unbreaking locks. How can I make these measurements from outside the power supply? I get those two numbers that they won't give us." And I did that. And it turns out we need a PSR measurement. We needed an impedance measurement. If we could do those two, we can get these magic numbers that we need to make this model go.

Ben Dan comes along and it says, "I really need to use the state space model. Can you walk me through the process because I'm reading this stuff and it just seems like there's leaps that I just don't get. And maybe you just didn't give us that brain fragment, but I need to get that brain fragment out of you." And so I spent some time with Ben Dan, and he worked his way through the model and he's now, I mean, he is an expert at this. And he said, "We need to do this as a paper because there's so many engineers that would love this model, but they're intimidated by the process and the process isn't really well documented. Let me document that process." And so that's what we're doing in the end of VR mode.

It is a lot of equations, but you don't really need those. The equations are all behind the scenes, right? All you need to do is plug in the right parameters. Turns out there's only two parameters that we really need to know. And if you can get those two parameters, the model's perfect, then Ben is going to show that.

Zach Peterson:

So a few things. So first, just for perspective, if you were to do this in spice, it would take on the order of many seconds. This happens under a second. I'm going to assume that this is because state space model is not an iterative process the way you would do it in spice.

Steve Sandler:

Correct. And actually we could do this in spice now. In fact, P SPICE has this model. The whole point is that we need to figure out what it is that the power supply is going to do in the time domain because it's a time domain switch. We need to figure out what that looks like in a continuous equation. And it turns out it's not that hard. If you think about the switching power supply, they're only two points that we know. I mean, it's like when we look at a telescope, we see these curves, but it's not really curves. It's a bunch of points. And the switching power supply, there's only two points. The beginning of a cycle and the end of a cycle. Everything between those two is interpolated. So the state space model says, "All right, if I know when the start of a cycle is, I'm going to do, because I know what the frequency of it is, and I know what the end of a cycle is a do because it's the duty cycle."

And that is an output of the state space parameter. State space parameter says, "I got one volt coming out, five volts in, and that means I got a 20% duty cycle." And so I know the width of this thing and of the start. I know the stop and everything else I can interpolate. The state space model fills in all of those blanks in a continuous equation that says, "Okay, now if I  these things, I know it's going to happen in the continuous equation. And if I know what it's doing in the continuous equation, I can give you those two time points." And the difference between that and running a P spice model is that in spice, I have to solve voltage and current equations, and I need to pick the time step. The time step needs to be fast enough for the switching frequency.

So let's say I pick a 10 nanosecond time step or a hundred nanosecond time step, and I need to simulate a millisecond. Simulating millisecond at a 10 nanosecond time step. I mean, that's painful, right? Thousands and thousands of voltage and current equations. Whereas the state space model says, "Okay, so you want to simulate a hundred nanosecond interval and you want to simulate 10 milliseconds, that's fine. I just need to figure out what the states are, and I'm going to simulate two points for every cycle. I'm not going to simulate thousands of points. I'm going to simulate two start of a cycle, end of a cycle, and I'm going to interpolate those just like the power supply will in between those points." That's the way that this works. That's why it's so fast. Only need to simulate those two points. Start and stop.

Zach Peterson:

Innovative is how I would describe it. But the other thing you had mentioned, I think you had said that you're modulating the duty cycle in this idea. By modulating the duty cycle, do you mean the amplitude, the-

Steve Sandler:

No, the actual time. Yeah, the actual time. We're modulating pulse width and we're seeing how that modulation affects the power supply outputs. We're also looking at how that affects the power supply inputs. So what would happen if I the duty cycle and looked at the ripple? Or conversely, what if I modulated the input voltage? How would that affect the duty cycle? And if that affected the duty cycle, how does that affect everything else? So, I have a bunch of papers on why you need to fully characterize models because it has so many noise ports and everyone affects every other one. And the state space model accounts for all of those. But just like everything else, I mean, I wrote this model, like I said 30 years ago. There's interest in it now.

Zach Peterson:

But at the end of the day, the goal here is we want to determine control loop behavior without actually knowing how the control loop works. So, we can apply these phenomenological perturbations to the output of the control loop, which is essentially just control over the duty cycle.

Steve Sandler:

Yes.

Zach Peterson:

I see. Okay. And so then-

Steve Sandler:

Well, once we figured that out, we've unlocked the key to everything, right?

Zach Peterson:

Assured because then at that point you could sweep through different, I guess, magnitude of perturbations on those parameters and really see how the circuit responds.

Steve Sandler:

Yeah. In fact, what's interesting in Ben's paper is that he synthesized the whole power supply by these two measurements, came up with the state space model parameters, and there was one problem that he had. There was a little ringing in the ripple that he couldn't figure out, and he went back and he included the printed circuit board, and he realized that it was a resonance on the printed circuit board. Between-

Zach Peterson:

Always a resonance on the printed circuit board.

Steve Sandler:

On the ceramic capacitors, and he added the printed circuit board and matched the ripple perfectly.

Zach Peterson:

Very cool. Very cool. Well, we're up.

Steve Sandler:

We're out of time.

Zach Peterson:

Yeah. Yeah. We're getting up there on time, but I hope that folks that are listening will head over to DesignCon and take a look at all of these presentations because this is also interesting. And me being a math and simulation guy and a physicist, I eat this stuff up. So I love hearing about this. So, thank you very much for coming on and talking with us today.

Steve Sandler:

Well, thanks so much for having me. It was fun. And I hope to see a lot of you guys at DesignCon. For me, that is like... That's the year, special time, DesignCon.

Zach Peterson:

Yeah, absolutely. To everyone that's been listening, we have been talking with Steve Sandler, founder and managing director of PECO Test. We're going to have some great links in the show notes, including some links to the papers that Steve mentioned. I encourage you all to go take some time to read those papers and learn more about everything that we've been talking about in this podcast. Make sure to subscribe if you're watching on YouTube, you'll be able to keep up with all of our upcoming tutorials and podcast episodes. And last but not least, don't stop learning. Stay on track, and we'll see you next time.