database - Performance of foreach loop In C#

Question

I am working on C# API's which gets data from database. I inserted around 19500 records in database for testing performance. Here, sampledData contains 19500 records.

var sampledData = await _dataContext.ItemsData
                                    .Include(i => i.ProcedureItem)
                                    .Include(i => i.ProcedureItem.ProcedureItem)
                                    .Include(i => i.ProcedureItem.ProcedureItemAll)
                                    .Where(i => i.Procedure.Status == true &&
                                                i.isValid== true &&
                                                i.Procedure.ID== ID).ToListAsync();

// Foreach loop on 19500 records to filter data and store them info list
var filteredList = new List<ProcedureFilteredData>();

foreach(var s in sampledData )
{
    if (filteredList.Any(i => i.ProcedureItem == s.ProcedureItem.ProcedureItem.Name))
    {
        continue;
    }

    var pData = new ProcedureFilteredData
    {
        ProcedureItemAll = s.ProcedureItem.Name
    };

    filteredList.Add(pData);
}

In filteredList, I get 1404 records. It takes time in foreach loop to filter data. Is there any way to optimize performance?

Answer 1

I suggest checking with a help of HashSet<T>, not List<T> since filteredList .Any(...) has quadratic (O(n**2)) time complexity while !knownNames.Add(...) is linear (O(n)):

HashSet<string> knownNames = new HashSet<string>();

var filteredList = new List<ProcedureFilteredData>();

foreach (var s in sampledData)
{
    if (!knownNames.Add(s.ProcedureItem.ProcedureItem.Name))
        continue;  

    var pData = new ProcedureFilteredData
    {
        ProcedureItemAll = s.ProcedureItem.Name
    };

    filteredList.Add(pData);
}