Thanks for writing!

The main thing I'm hearing in what you wrote is:

Bug: protected Stream functions like timedRead and/or the _timeout property are not defined

I'm changing the issue title to this, and I'd welcome any info you have on how you'd expect these functions to work during testing (as I don't have much personal experience with the deeper parts of Stream).

As far as the need for Stream (as well as all of the mocked classes I provide in this project) is that Arduino's libraries were written for an AVR backend and simply will not compile at all on any other architecture.

As far as testing, you're right -- in a perfect world people would use dependency injection and isolate themselves from low-level components. But the Arduino ecosystem was built without testing in mind, so between the hard-coded assumption that you will have a setup() and loop() entry point and the fact that you can only compile on AVR architecture really limited my options.

Even the idea of mocking the low-level boards support (i.e. your target architecture is a software emulator) is problematic, because then you have to write one mock per board type.

I still hope against hope that the Arduino folks will push for more than compilation tests. The Adafruit team seems to be driving that effort and "unit tests" don't seem to be in their vocabulary. I've mentioned unit testing a few times on the developer mailing list and get absolute silence in response.

The Stream works just fine if you just copy the existing original Stream.h and Stream.cpp from the Arduino core. This is what I did in my own fork. There should be no need to modify the Stream at all and it just causes problems. If you really need to mock for example the HardwareSerial do it just there and leave the Stream as it is.

The non virtual functions in the Stream only use the virtual functions that are implemented in the derived classes. There is not much depencies to other Arduino code there.

The project that I'm testing using this is here: https://github.com/lasselukkari/aWOT

I received some helpful information that might be relevant to this issue. At the very least, I can see that my reverse-engineering of the IDE's g invocation wasn't totally accurate. (Compare the listing in that issue comment with the flags I use, -std= and -O1 in particular.)

I think the proper thing for me to do here is take a step back and try to rebuild the mocks based on the _ correct_ set of compiler flags. @lasselukkari is that something you'd be interested in being involved with?

I believe that the proper approach here would be to keep Stream completely unmodified compared to the Arduino version, since it's just an abstract superclass that can compile on a PC just fine. Then any mocking should be done in subclasses. Something like a MockedStream class that extends Stream and is again extended by HardwareSerial would make sense. It seems there is already TapedStream class that occupies this place, but instead of being implemented on top of the standard Arduino Stream class (by just implementing the read and write methods), it seems to be heavily coupled to changes in the Stream class.

Also see #145 for some thoughts on further decoupling the mocking behaviour form the object being mocked.

I've opened #146 ... I should have opened it sooner.

The Arduino project has moved the common pieces to a shared repo: https://github.com/arduino/ArduinoCore-API. IMO these files should be taken there and left as unmodified as possible.

I see that you are working hard currently to add new features. Thanks for that. But I would like to see more simplification and clean up than features. I would like to contribute to the project but unfortunately I do not agree with the general direction at the moment. I think there should be a more clear separation with the Arduino core and the testing related functionality.

It's funny that you bring this up, because I've been looking at revisiting the entire set of C mocks. The feeling of "overcomplicating it" was exactly what I thought of when looking at an "official" Arduino.h.

In the beginning I was just creating compilation mocks as fast as I could and brute-forcing my way through compiler errors. Things like the hardware need to be mocked, but everything else can/should just use the existing libraries as much as possible! I think the only snag might be pgmspace.h and how it interacts with String and whatnot. I can't remember exactly how that shook out.

Big picture is that I need to create viable mocks for each platform. For example, what avr-libc provides, which is the general topic of #241 and #251

I do not agree with the general direction at the moment

What do you perceive that direction to be?

What do you perceive that direction to be?

Too many built in features. I may be wrong. I just quickly took a look at the commits since last spring.

Personally I would be happy to implement the mocks in my own to the project I'm testing when needed instead of getting everything built in or at least that I could control when they are included. Yes, millis and micros should advance by default and it's needed that people can control the return value of the wall clock but I do no need the "godmode".

I would personally only need the files from Arduino CoreApi, a test test runner, basic assertions and a way to compile the examples for selected target platforms. If everything else would be optional I would be really happy. I do understand that in the projects I'm currently testing with this it's easy to abstract the hardware away. This may not always be the case for libraries that interact with the hardware more directly. I just hope that for example I would not have to deal with the built in mocked Stream class. Instead in my own project I have this MockStream.h It is not a complete solution for all use cases but something like that would be better than directly mocking the Stream, Client or any of the other files that come from the CoreApi.

I happen to have time to contribute to open source at the moment and I would be pappy to help with this project too. The problems is that in the current state I can not use this and had to do my own fork.

PS. I should lear to write my comments complete first and only then hit the send button...

I'm sorry that the journey of this project has frustrated you to the point of wanting to abandon it. I think you and I agree on the need to reduce the complexity of the mocks, and I am certainly planning to revisit a lot of my previous design decisions now that ArduinoCore-API is available (in a way that it wasn't at the time I started writing this).

I would absolutely prefer to remove a good chunk of C code from this project if I could fall back on a more official version of it. The only things I want to have mocks for concern the clock and IO devices (pins, SPI, serial, etc) -- to be able to set up deterministic sequences of inputs for repeatable testing. And the only strong opinion that I have is this: for an existing library to support unit testing, it should not need to be completely rewritten (which seems only fair, since most of the popular resources on how to write libraries do not suggest using unit-testable design patterns). Arduino is aimed at a group of users who may be programming for the first time, after all.

I can think of 2 possible ways to reconcile my advocacy for "godmode" with your preference to use non-mocked libraries by default and inject your own custom mocks where desired.

The first option would be to allow (in the unit tests) a different #include option -- rather than #include <Arduino.h>, we could make it possible to include a file with a different name that offers non-mocked setup. Or a config file option? Anything that will satisfy the compiler seems reasonable to me.

The second option, which I'd prefer, (and where I'd like to hear perspectives of Arduino contributors like @matthijskooijman and @per1234) is for ArduinoCore-API to take on features that support unit testing or function mocks. In this setup, there would be some API function (or functions) exposed by Arduino that would essentially allow dependency injection for all the builtin functions. Is that kind of suggestion a non-starter?

@ianfixes have you taken a look at the ArduinoCore-API unit tests?
https://github.com/arduino/ArduinoCore-API/tree/master/test/src

ArduinoCore-API is all the hardware agnostic code of the core library so it shouldn't really require much mocking. Where it is necessary, there are probably already some of the mocks written because the coverage is up to 96% now.

I'll admit that I'm not 100% clear on how something like their HardwareSerial.h could be used for mocking purposes without also dragging in the hard-coded Stream reference.

@per1234 can you comment on my assessment of being unable to mock HardwareSerial without mocking Stream?

The truth is that, even though I have some experience with unit tests and mocking in other contexts, I really don't understand much of what is happening in this project. I was hoping to pick up some understanding by osmosis, but I've been watching this repo for years now without getting very far.

I can honestly say the same about the Arduino C infrastructure for boards and platforms 😉

What's the right forum where I could engage with the Arduino folks for a deeper dive on this sort of thing?

I think probably you are already using the available communication channels so I don't have anything else to suggest.

Arduino forum: Arduino users, but mostly not monitored by anyone from Arduino other than myself.

Arduino developers mailing list: A subset of the 3rd party developers. Seems to be fairly well monitored by Arduino, even when not actively participating in the conversations.

There is an official Discord server: https://discord.gg/jQJFwW7. I have only taken a quick look at it so I can't say whether it would be of any use to you.