Is SwiftData not designed for large datasets? How to create many instances of my Data Model?
I have a loop where I iterate over a CSV file of 30k+ lines. Totalling around 230k elements. I intend to save this data into SwiftData. However it takes around 90 seconds. In previous versions of my app I was using coreData. Then it took just 3 seconds. So after much investigation I have concluded the delay comes from creating instances of the Data Model. To Test this I have :
-
Removed any saves into SwiftData. So only creation of the Model Object occurs.
-
Created an exact copy of the SwiftData Model class but admitted the @Model macro
-
set isAutosaveEnabled to false
When I run my loop with the copy class it takes 3 seconds. So It must be creating the SwiftData model instances that causes the *30 delay.
The code using swiftData @Model. This loop takes 90+ seconds to complete
class TimetableData { var arrivalTime : Int = 0 var departureTime: Int = 0 var departureRoute : Int = 0 var directionOfTravel : Int = 0 var dutyNumber : Int = 0 var facilityIdentifier : String = "" var locationType : String = "" var mode : String = "" var nonStopStatus : Int = 0 var recordIdentifier : String = "" var timetableIdentifier : Int = 0 var trainNumber : String = "" var tripNumber : Int = 0 var tripStartSite : String = "" var uniqueLocationCode : String = "" init(arrivalTime: Int = 0, departureTime: Int = 0, departureRoute: Int = 0, directionOfTravel: Int = 0, dutyNumber: Int = 0, facilityIdentifier: String = "", locationType: String = "", mode: String = "", nonStopStatus: Int = 0, recordIdentifier: String = "", timetableIdentifier: Int = 0, trainNumber: String = "", tripNumber: Int = 0, tripStartSite: String = "", uniqueLocationCode: String = "") { self.name = name self.arrivalTime = arrivalTime self.departureTime = departureTime self.departureRoute = departureRoute self.directionOfTravel = directionOfTravel self.dutyNumber = dutyNumber self.facilityIdentifier = facilityIdentifier self.locationType = locationType self.mode = mode self.nonStopStatus = nonStopStatus self.recordIdentifier = recordIdentifier self.timetableIdentifier = timetableIdentifier self.trainNumber = trainNumber self.tripNumber = tripNumber self.tripStartSite = tripStartSite self.uniqueLocationCode = uniqueLocationCode } } for line in CSVFile { var columnValue = line.components(separatedBy: ",") let timeTableDataEntry = TimetableData ( arrivalTime: Int(columnValue[0])!, departureTime: Int(columnValue[1])!, departureRoute: Int(columnValue[2])!, directionOfTravel: Int(columnValue[3])!, dutyNumber: Int(columnValue[4])!, facilityIdentifier: columnValue[5], locationType: columnValue[6], mode: columnValue[7], nonStopStatus: Int(columnValue[8])!, recordIdentifier: columnValue[9], timetableIdentifier: Int(columnValue[10])!, trainNumber: columnValue[11], tripNumber: Int(columnValue[12])!, tripStartSite: columnValue[13], uniqueLocationCode:columnValue[14] ) }
Then The exact same code but omitting the @Model takes around just 3 second.
class TimetableDataForTesting { var arrivalTime : Int = 0 var departureTime: Int = 0 var departureRoute : Int = 0 var directionOfTravel : Int = 0 var dutyNumber : Int = 0 var facilityIdentifier : String = "" var locationType : String = "" var mode : String = "" var nonStopStatus : Int = 0 var recordIdentifier : String = "" var timetableIdentifier : Int = 0 var trainNumber : String = "" var tripNumber : Int = 0 var tripStartSite : String = "" var uniqueLocationCode : String = "" init(arrivalTime: Int = 0, departureTime: Int = 0, departureRoute: Int = 0, directionOfTravel: Int = 0, dutyNumber: Int = 0, facilityIdentifier: String = "", locationType: String = "", mode: String = "", nonStopStatus: Int = 0, recordIdentifier: String = "", timetableIdentifier: Int = 0, trainNumber: String = "", tripNumber: Int = 0, tripStartSite: String = "", uniqueLocationCode: String = "") { self.name = name self.arrivalTime = arrivalTime self.departureTime = departureTime self.departureRoute = departureRoute self.directionOfTravel = directionOfTravel self.dutyNumber = dutyNumber self.facilityIdentifier = facilityIdentifier self.locationType = locationType self.mode = mode self.nonStopStatus = nonStopStatus self.recordIdentifier = recordIdentifier self.timetableIdentifier = timetableIdentifier self.trainNumber = trainNumber self.tripNumber = tripNumber self.tripStartSite = tripStartSite self.uniqueLocationCode = uniqueLocationCode } } for line in CSVFile { var columnValue = line.components(separatedBy: ",") let timeTableDataEntry = TimetableDataForTesting ( arrivalTime: Int(columnValue[0])!, departureTime: Int(columnValue[1])!, departureRoute: Int(columnValue[2])!, directionOfTravel: Int(columnValue[3])!, dutyNumber: Int(columnValue[4])!, facilityIdentifier: columnValue[5], locationType: columnValue[6], mode: columnValue[7], nonStopStatus: Int(columnValue[8])!, recordIdentifier: columnValue[9], timetableIdentifier: Int(columnValue[10])!, trainNumber: columnValue[11], tripNumber: Int(columnValue[12])!, tripStartSite: columnValue[13], uniqueLocationCode:columnValue[14] ) }