use the following search parameters to narrow your results:
e.g. subreddit:aww site:imgur.com dog
subreddit:aww site:imgur.com dog
see the search faq for details.
advanced search: by author, subreddit...
573 users here now
/r/programming is a reddit for discussion and news about computer programming
Please try to keep submissions on topic and of high quality.
Just because it has a computer in it doesn't make it programming.
Memes and image macros are not acceptable forms of content.
If there is no code in your link, it probably doesn't belong here.
App demos should include code and/or architecture discussion.
Please follow proper reddiquette.
Do you have a question? Check out /r/learnprogramming, /r/cscareerquestions, or stackoverflow.
Do you have something funny to share with fellow programmers? Please take it to /r/ProgrammerHumor/.
For posting job listings, please visit /r/forhire or /r/jobbit.
Check out our faq. It could use some updating.
If you're an all-star hacker (or even just beginning), why not join the discussion at /r/redditdev and steal our reddit code!
Ask Reddit: Machine Learning Question: (programming.reddit.com)
submitted 8 years ago by [deleted]
[–]api 7 points8 points9 points 8 years ago (0 children)
Theoretically, no. The constant stream of "Bush sucks" articles on Reddit doesn't hinder my ability to notice one about machine learning. :)
But I can see how some algorithms might have this problem due to algorithmic artifacts. It would really depend on precisely what kind of neural network you're using.
[–][deleted] 3 points4 points5 points 8 years ago (3 children)
In general, does adding redundant features to a machine learning algorithm hinder it?
For example, suppose i add an input to a neural network that is constant 95% of the time and only very occasionally gives some useful info. Will this hinder the algorithm more than it helps it?
[–]mrned 1 point2 points3 points 8 years ago (0 children)
You'd have to define hinder more precisely. If by hinder you mean reduce accuracy, then the answer depends on the algorithm you choose; some don't lose accuracy with extraneous features, some do. In general, though, your learning or classification run will take longer because you've got larger dimensions than you need.
[–]ecuzzillo 0 points1 point2 points 8 years ago (0 children)
In general, it is clear that if you give an absolutely constant feature to a learning algorithm, it won't change anything. In the case of the neural network, you already have things like this-- they're called bias inputs, and they're what you use to make sure your network can represent the fact that the outputs might not be zero when all the inputs are zero. So if you add another bias input, the algorithm that learns the weights for the neural network will adjust the two bias weights to act as if you had only one bias. I suppose, however, that if you had forgotten to give a bias input to your network, and you had biased data, that adding a constant feature would help your performance. That's mainly because it increases your set of concepts (aka possible representable functions) to include ones that more accurately fit the data.
Furthermore, although I don't know enough to say this rigorously, usually it is the case that the limit as the feature goes to a constant is just the same as if you didn't have the feature at all. Look up PAC Learning Theory in your favorite machine learning textbook for a better treatment.
[–]schwarzwald 0 points1 point2 points 8 years ago (0 children)
i'm pretty sure the neural net would extract the fact that your constant input has essentially no bearing on the output and would basically ignore it. thus the weight associated with it would approach zero, i think. although i could be wrong.
if you're afraid of having too large a feature set, there are feature selection algorithms out there, e.g. genetic algorithms that try to select optimal feature subsets.
[–]timclark 0 points1 point2 points 8 years ago (1 child)
If you are training your neural network using something like back propagation I would think it would depend on the training sets that you present to the network. If you are using something like a self organizing map I guess it might eventually notice.
When I used to work alongside speech researchers they would do their utmost to reduce redundancy in the features they were examining before presenting them to their neural network, hidden markov model etc. We only had small computers then and it definitely speeded up training.
well, popular techniques for producing uncorrelated inputs like principal component analysis also basically double as dimensionality reduction tools
[–]palmereldritch 0 points1 point2 points 8 years ago (0 children)
This is known as the curse of dimensionality. Despite claims to the contrary (e.g. for support vector machines), I don't think anyone has beaten it.
[–]hadley 0 points1 point2 points 8 years ago (1 child)
I don't think you can make a statement in general about all machine learning algorithms. Adding redundant features is never going to help, but depending on the algorithm it may be not too bad or a complete disaster. It tends to be more of a problem for clustering (unsupervised learning) because the definition of a redundant feature is circular: a redudant feature doesn't help distinguish the clusters, but until you have the clusters you can't determine what features are redundant.
Perhaps you could provide more information about the algorithm you are considering.
Yes, you can make statements in general about all machine learning algorithms. It's called machine learning theory, and it's a reasonably-well-studied field, with many interesting people (such as Avrim Blum) studying it currently.
REDDIT and the ALIEN Logo are registered trademarks of reddit inc.
π Rendered by PID 30803 on app-24 at 2015-02-28 05:18:14.236349+00:00 running fc50792 country code: US.