Spring 2020 Ebike Reliability Survey Results

[EbikeTestLab]

yes, you can take it here:

Ebike Reliability Survey

ebiketestlab.survey.fm

I should have added that any future survey responses will be included in the next report, which will be out this Fall. I may add a few questions and make some other small changes to the survey between now and then but I can still include responses before the survey change.

 

[Phyz]

I promise I looked. But I didn't see the total number of surveys, and a breakdown of surveys / brand. Do I need more coffee?

 

[EbikeTestLab]

I promise I looked. But I didn't see the total number of surveys, and a breakdown of surveys / brand. Do I need more coffee?

I always need more coffee! Sorry, I added that to the thread on Reddit but forgot here. There were 542 responses in total. Most brands on the list had 20 to 30 respondents.

 

[Alex M]

It often happens that the people who spend more have more complaints, just because they have higher expectations.

True. 3-star hotels are sometimes getting less complaints than 5-star ones, because people have lower expectations - all is good if there are no bedbugs and roof isn't leaking.

Still doesn't explain why Juiced (for example) fared much better in this review than Aventon or RAD, and even better than expensive Giant or BH. A glitch perhaps?

 

[Phyz]

I always need more coffee! Sorry, I added that to the thread on Reddit but forgot here. There were 542 responses in total. Most brands on the list had 20 to 30 respondents.

Keep up the good work; objective data is scarce in this field. It's not always appreciated by everyone—especially when it runs counter to their strongly-held personal opinions. Some folks prefer that their opinions and anecdotal experiences go unchallenged.

But it is appreciated by this of us who value objectivity. As you know, more data is better, and longitudinal data is better-er.

 

[Phyz]

A glitch perhaps?

Objective data can be surprising where subjectivity has been the sole ruler. That only speaks to the value of collecting it, and then collecting more of it.

 

[Alex M]

Objective data can be surprising where subjectivity has been the sole ruler. That only speaks to the value of collecting it, and then collecting more of it.

That also speaks to the value of processing the data.
Collecting more is always a good idea, makes it easier to filter out results that are not statistically significant.

 

[pmcdonald]

Outstanding, thank you for compiling this! You've been upfront about the issues and really prompt with providing updates so, again, thank you. We need more info like this.

 

[Phyz]

That also speaks to the value of processing the data.
Collecting more is always a good idea, makes it easier to filter out results that are not statistically significant.

As long as you don’t filter out results on the basis of “that result doesn’t conform to my expectation, so the data must be wrong.”

 

[EbikeTestLab]

Where is the breakdown by component?

I will add it in above in the original post, thanks for the reminder. I had included it on the Reddit post but forgot to do so here.

 

[6zfshdb]

I should have added that any future survey responses will be included in the next report, which will be out this Fall. I may add a few questions and make some other small changes to the survey between now and then but I can still include responses before the survey change.

I have three bikes, all are the same make and model. I originally filled out the survey based on my experience with the one I ride most. Would filling out the survey again for the other two bikes be of any value considering they are all the same brand?

 

[EbikeTestLab]

I have three bikes, all are the same make and model. I originally filled out the survey based on my experience with the one I ride most. Would filling out the survey again for the other two bikes be of any value considering they are all the same brand?

I have now limited the survey to only one survey per device in a effort to prevent any fraudulent respondents. However, you could certainly take the survey with a different device for each ebike you own. The more data, the better.

To be clear, I don’t believe I had any fraudulent responses in this survey but a weakness was pointed out to me last night and this change makes sense moving forward.

 

[Marci jo]

Thank you for assembling this data. It’s interesting.
I’m thinking it would be statistically more valuable if the total number of each bike sold could be included. Too bad the manufacturers don’t publish that data.

 

[EbikeTestLab]

Thank you for assembling this data. It’s interesting.
I’m thinking it would be statistically more valuable if the total number of each bike sold could be included. Too bad the manufacturers don’t publish that data.

Agreed! It would certainly help normalize the data.

 

[Nxkharra]

Great subject. Thanks to all.

 

[Alex M]

As long as you don’t filter out results on the basis of “that result doesn’t conform to my expectation, so the data must be wrong.”

Please look up the term "statistically significant", as it appears that you don't understand it.

 

[Admin]

Hey @EbikeTestLab cool results! I wonder if some of the major brands saw higher failure rates communicated because there was a higher percentage of users who bought those bikes in general? Like, was the survey sent to an even number of owners for all brands, or is this self-reported? I'd like to know the percentage breakdown of owner per bike... like if there was only a few Riese & Müller owners, but a bunch of Rad, Aventon, Specialized, that could skew the results.

 

[EbikeTestLab]

Hey @EbikeTestLab cool results! I wonder if some of the major brands saw higher failure rates communicated because there was a higher percentage of users who bought those bikes in general? Like, was the survey sent to an even number of owners for all brands, or is this self-reported? I'd like to know the percentage breakdown of owner per bike... like if there was only a few Riese & Müller owners, but a bunch of Rad, Aventon, Specialized, that could skew the results.

Thanks Court! Just from memory, most of the brands had 20 to 30 surveys. I can break out the actual numbers once I get a chance. The dataset is somewhat limited with only 542 responses so the data could be skewed due to the limited data set. I don’t recall any one brand having an unusually low or high amount of reviews, other than WattWagons as stated in the original post.

I’ll get back to you on the breakout by brand.

 

[Ravi Kempaiah]

Thanks Court! Just from memory, most of the brands had 20 to 30 surveys. I can break out the actual numbers once I get a chance. The dataset is somewhat limited with only 542 responses so the data could be skewed due to the limited data set. I don’t recall any one brand having an unusually low or high amount of reviews, other than WattWagons as stated in the original post.

I’ll get back to you on the breakout by brand.

More data is always beneficial, something that most scientists look for is solid data.

A more useful approach would be to populate data of 50 or 100 users for each brand and then segment it into different component failure and then start the analysis.
If you would like to gain name and credibility as a source of reliable information, then providing the source of data with statistical analysis in the form of histograms or charts would be helpful.
This provides the readers deeper perspective, not just a number.

It would be wise to populate baseline data points for each brand, no need to rush.

 

[EbikeTestLab]

More data is always beneficial, something that most scientists look for is solid data.

A more useful approach would be to populate data of 50 or 100 users for each brand and then segment it into different component failure and then start the analysis.
If you would like to gain name and credibility as a source of reliable information, then providing the source of data with statistical analysis in the form of histograms or charts would be helpful.
This provides the readers deeper perspective, not just a number.

It would be wise to populate baseline data points for each brand, no need to rush.

Thanks for the tips Ravi! I’m used to working in much larger datasets in my day job as a mechanical engineer. Due to the limited dataset, the amount of stats possible was limited. I still feel that this data is valuable information for folks interested in ebike reliability.
From my day job, I regularly present complex data sets to folks that aren’t engineers. They need a simple result to understand, at least at first. Some folks, like you and me, would prefer the complete dataset so you could make your own evaluations. Most folks aren’t like that though and I wanted to present a ranking and mileage per failure that would be easily understood.

Perhaps with my next survey, I’ll have enough responses to populate 100 or so for each brand and start the analysis there. Thanks again, with these suggestions, the survey will improve continuously with each iteration.