Alex Neil said ministers would also look to reduce the stress of applying for social security.
Responsibility for a number of benefits is being devolved to MSPs as part of the new Scotland Bill.
They include the Carer's Allowance and benefits for the disabled such as the Disability Living Allowance.
Just over 15% of welfare spending north of the border will be devolved, with the legislation also giving the parliament in Edinburgh the power to create new benefits and top up existing payments.
Mr Neil will use a debate on Tuesday to set out the key principles that will underpin the Scottish government approach to welfare - that social security is an investment in the people of Scotland, with respect for the individual at the heart of the system.
Holyrood ministers also stress welfare services will be "evidence based", and provide value for money, while "putting people first" will be central to all policies, processes and systems.
Mr Neil said: "With our new social security powers we have the opportunity to take a different approach and develop policies for Scotland which will help to remove the stigma attached to accessing benefits.
"These policies will be based on principles which will ensure people are treated with dignity and respect. We want to show that social security can be fairer, tackle inequalities, and protect and support the vulnerable in our society."
"It's understandable that people who receive benefits, who are ill, retired or disabled, will be concerned about how these changes will affect them.
"I want to reassure them that our priority is for them to receive their benefits on time and with the right amount."
He added: "We want to ease some of the stress of applying for benefits and will make the system easier and simpler to navigate, and align it with our devolved services that support people."
The Scottish government wants to use new welfare powers to help remove the stigma attached to claiming benefits, the social justice secretary has said.